The Importance of Cache

Hi!

I recently posted a blog entry on data orientation, where one of the consequences of following this design mindset was a more optimal cache utilization.

Intrigued by the claims about how the memory is becoming the main bottleneck in current systems, I wanted to test it by myself, so I came up with a very simple C++ program.

#include <iostream>

#include <chrono>

const unsigned long int SIZE= 64*1024;

int main()

{

char array[SIZE];

char array2[SIZE];

auto start = std::chrono::system_clock::now();

for(int i = 0; i < 1000; ++i)

{

int aux = array[i];

}

auto duration = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::system_clock::now() - start);

std::cout << "Duration: " << duration.count() << std::endl;

auto start2 = std::chrono::system_clock::now();

for(int i = 0; i < 1000; ++i)

{

int aux = array2[i*64];

}

auto duration2 = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::system_clock::now() - start2);

std::cout << "Duration 2: " << duration2.count() << std::endl;

double res = (double)(duration2.count()) / (double)(duration.count());

std::cout << "First is " << res  << " faster" << std::endl;

}

I simply define and read two arrays of 64 KBs each, storing their values in a local variable. The difference is that in the first chunk of code I access memory sequentially, taking advantage of the cache, whereas in the second chunk I access in strides of 64 bytes, which in turn causes cache misses and more frequent access to main memory. 

The results are astonishing: the first chunk (the cache-friendly code) executes an average of 7.3 times faster. So yes! Memory is an important bottleneck in today's systems. 

See you!

FM