I got really strange behavior for my x64 multithreading application. The execution time in debug mode is faster than in release mode.
I break the problem down and found the issue: The debug modus optimize (!Note optimition is off!) the memcpy to memmove, which peforms faster. The release mode use still memcpy (!note optimition is on).
This problem slows down my multithreading app in release mode. :(
Anyone any idea?
#include <time.h> #include <iostream> #define T_SIZE 1024*1024*2 int main() { clock_t start, end; char data[T_SIZE]; char store[100][T_SIZE]; start = clock(); for (int i = 0; i < 4000; i++) { memcpy(store[i % 100], data, T_SIZE); } // Debug > Release Time 1040 < 1620 printf("memcpy: %d\n", clock() - start); start = clock(); for (int i = 0; i < 4000; i++) { memmove(store[i % 100], data, T_SIZE); } // Debug > Release Time 1040 > 923 printf("memmove: %d\n", clock() - start); }
The following answer is valid for VS2013 ONLY
What we have here is actually stranger than just memcpy vs. memmove. It's a case of the intrinsic optimization actually slowing things down. The issue stems from the fact that VS2013 inlines memcopy like thus:
; 73 : memcpy(store[i % 100], data, sizeof(data)); mov eax, 1374389535 ; 51eb851fH mul esi shr edx, 5 imul eax, edx, 100 ; 00000064H mov ecx, esi sub ecx, eax movsxd rcx, ecx shl rcx, 21 add rcx, r14 mov rdx, r13 mov r8d, 16384 ; 00004000H npad 12 $LL413@wmain: movups xmm0, XMMWORD PTR [rdx] movups XMMWORD PTR [rcx], xmm0 movups xmm1, XMMWORD PTR [rdx+16] movups XMMWORD PTR [rcx+16], xmm1 movups xmm0, XMMWORD PTR [rdx+32] movups XMMWORD PTR [rcx+32], xmm0 movups xmm1, XMMWORD PTR [rdx+48] movups XMMWORD PTR [rcx+48], xmm1 movups xmm0, XMMWORD PTR [rdx+64] movups XMMWORD PTR [rcx+64], xmm0 movups xmm1, XMMWORD PTR [rdx+80] movups XMMWORD PTR [rcx+80], xmm1 movups xmm0, XMMWORD PTR [rdx+96] movups XMMWORD PTR [rcx+96], xmm0 lea rcx, QWORD PTR [rcx+128] movups xmm1, XMMWORD PTR [rdx+112] movups XMMWORD PTR [rcx-16], xmm1 lea rdx, QWORD PTR [rdx+128] dec r8 jne SHORT $LL413@wmain
The issue with this is that we're doing unaligned SSE loads and stores which is actually slower than just using standard C code. I verified this by grabbing the CRTs implementation from the source code included in with visual studio and making a my_memcpy
As a way of ensuring that the cache was warm during all of this I had preinitialized all of data but the results were telling:
Warm up took 43ms
my_memcpy up took 862ms
memmove up took 676ms
memcpy up took 1329ms
So why is memmove faster? Because it doesn't try to prior optimize because it must assume the data can overlap.
For those curious this is my code in full:
#include <cstdlib> #include <cstring> #include <chrono> #include <iostream> #include <random> #include <functional> #include <limits> namespace { const auto t_size = 1024ULL * 1024ULL * 2ULL; __declspec(align(16 )) char data[t_size]; __declspec(align(16 )) char store[100][t_size]; void * __cdecl my_memcpy( void * dst, const void * src, size_t count ) { void * ret = dst; /* * copy from lower addresses to higher addresses */ while (count--) { *(char *)dst = *(char *)src; dst = (char *)dst + 1; src = (char *)src + 1; } return(ret); } } int wmain(int argc, wchar_t* argv[]) { using namespace std::chrono; std::mt19937 rd{ std::random_device()() }; std::uniform_int_distribution<short> dist(std::numeric_limits<char>::min(), std::numeric_limits<char>::max()); auto random = std::bind(dist, rd); auto start = steady_clock::now(); // warms up the cache and initializes for (int i = 0; i < t_size; ++i) data[i] = static_cast<char>(random()); auto stop = steady_clock::now(); std::cout << "Warm up took " << duration_cast<milliseconds>(stop - start).count() << "ms\n"; start = steady_clock::now(); for (int i = 0; i < 4000; ++i) my_memcpy(store[i % 100], data, sizeof(data)); stop = steady_clock::now(); std::cout << "my_memcpy took " << duration_cast<milliseconds>(stop - start).count() << "ms\n"; start = steady_clock::now(); for (int i = 0; i < 4000; ++i) memmove(store[i % 100], data, sizeof(data)); stop = steady_clock::now(); std::cout << "memmove took " << duration_cast<milliseconds>(stop - start).count() << "ms\n"; start = steady_clock::now(); for (int i = 0; i < 4000; ++i) memcpy(store[i % 100], data, sizeof(data)); stop = steady_clock::now(); std::cout << "memcpy took " << duration_cast<milliseconds>(stop - start).count() << "ms\n"; std::cin.ignore(); return 0; }
Update
While debugging I've found that the compiler did detect that the code I copied from the CRT is memcpy, but it links it to the non-intrinsic version in the CRT itself which uses rep movs instead of the massive SSE loop above. It seems the issue is ONLY with the intrinsic version.
Update 2
Per Z boson in the comments it seems that this is all very architecture dependent. On my CPU rep movsb is faster, but on older CPUs the SSE or AVX implementation has the potential to be faster. This is per the Intel Optimization Manual. For unaligned data as rep movsb can experience up to a 25% penalty on older hardware. However, that said, it appears that for the vast majority of cases and architectures rep movsb will on average beat the SSE or AVX implementation.
Idea: call memmove, since it's fastest for your case.
