Benchmarking C/C++

This article describes benchmarking, and the quickest path to success. Why? Because you don’t want to spend time doing benchmarking but to verify performance over time, you have to do benchmarking…

A Practical Example?

Use a library for benchmarking. Yes you can use rdtsc() on your own, but don’t be stupid; there is a lot more to benchmarking than just measuring “elapsed time”. The Google Benchmark library is good for easily creating some benchmarks – and it has all the features you need, but none of the weird stuff that gets in your way. It is a C++ library, and you’ll need to link your final benchmark binary with -lbenchmark and -lpthreads using a C++ compiler.

Your first benchmark:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#include <stdlib.h>
#include "benchmark/benchmark.h"
void benchy(benchmark::State& state)
{
    srand( time(0) );
    while (state.KeepRunning()) {
        for(int i = 0 ; i < 1000; i++) {
            float x = rand() / ((float)rand());
            benchmark::DoNotOptimize(x);
        }
    }
}
/* run benchy, only report summary, and run 3x for accuracy
BENCHMARK(benchy)->ReportAggregatesOnly(1)->Repetitions(3);
BENCHMARK_MAIN()

Compile and run as follows:

1
g++ main.c -lbenchmark -lpthread && ./a.out

Conclusion

Google benchmark makes it easy to benchmark things, and get some decent results. Write a quick benchmark now, profit for every git commit from now. There’s no point in writing a benchmark once performance has become a problem – you’ve already thrown away the performance insight as the code was being developed: so just write a benchmark already.

Footnotes

[1] Preventing Optimization is an interesting topic, covered well here.
[2] Commit your perf data as JSON to keep track of performance over time?