3

I watch this YouTuber's mostly excellent videos on .NET features (let's call him Nick). He more often than not uses Benchmark.net to demonstrate the differences in performance of different approaches to programming problems.

The results this tool shows for statements or full methods that typically have quite a bit of logic in them are typically a couple of nanoseconds. Sometimes even less than a single nanosecond.

With my understanding of modern microprocessors I can't help but think this cannot be right.

Even if I take into account that the test will be executed multiple times (filling up the cache, pipelines, optimizing branch prediction results and all those goodies, likely rendering the results meaningless anyway), the numbers don't add up for me. A 3 GHz processor can, under optimal conditions, do a couple of operations per cycle, a cycle being 1/3 of a ns. That is still a far cry from any rudimentary C# statement which would take tens to hundreds of operations.

What is the deal with those numbers? What am I missing or how do they come up with those kind of results?

Improve this question
7
  • Would you mind to share some links to those videos?
    – Doc Brown
    Commented Sep 13, 2022 at 3:01
  • @DocBrown Here's his latest one, first benchmark at 4:30. What is measured here should be pretty quick but the results still seem ridiculous. youtu.be/d76WWAD99Yo
    – Martin Maat
    Commented Sep 13, 2022 at 3:52
  • 3
    The examples in the video relate to accessing auto-implemented properties of various C# objects, and invoking void methods which are empty. The CLR will inline these. For the sealed classes, the CLR can optimize the calls away entirely (and the benchmarking tool shows a warning). For the non-sealed classes, the CLR will have to do a tiny bit more bookkeeping, but I guess that loop unrolling within the benchmark harness reduces this to <1 cycle per invocation. But it's still benchmarking the cost of doing nothing, and the results are not particularly interpretable.
    – amon
    Commented Sep 13, 2022 at 12:30
  • 1
    I think Amon is right, the video benchmarks about measuring empty functions aren't useful, they may be giving a distorted impression, because they contain too much of the benchmark framework's overhead. If you want more meaningful results, make a benchmark of your own, but use methods dealiing with some real-world task.
    – Doc Brown
    Commented Sep 14, 2022 at 6:17
  • 1
    "The results this tool shows for statements or full methods that typically have quite a bit of logic in them are typically a couple of nanoseconds" can you provide an example other than the video linked above? I think we can all agree the numbers comparing the overheads of a virtual and non-virtual call are meaningless (the tool already tries to account for such overheads); other videos by the same guy seem to contain more sensible comparisons, with benchmarks that measure in the microseconds. benchmarkdotnet.org/articles/guides/how-it-works.html may be of interest.
    – VisualMelon
    Commented Sep 14, 2022 at 11:56

1 Answer 1

Reset to default
2

You can’t measure one execution of code in a meaningful way. But if something supposedly takes a nanosecond then you just do it a billion times and count the seconds, without any tools. If that’s too fast then you run it 10 billion times and count the seconds. If that is still too fast, the optimiser has optimised your code away :-)

Another possibility is automatic vectorising, and I once found a compiler that unrolled an empty loop eight times so instead of a billion iterations doing nothing it did 125 million iterations doing nothing. So my code said “one billion”, and it took 3/8ths of a billion cycles :-) (3 cycles for an empty loop).

Improve this answer
1
  • 1
    There surely are some useful hints in this answer. Nethertheless it does not seeem to refer to any specifics of this question.
    – Doc Brown
    Commented Sep 14, 2022 at 6:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.