Improving memory layout for parallel computing

Question

I\'m trying to optimize an algorithm (Lattice Boltzmann) for parallel computing using C++ AMP. And looking for some suggestions to optimize the memory layout, just found out

Answer 1

Current GPUs are notoriously depending about memory layout. Without more details about your application here are some things I would suggest you explore:

1. Unit-stride access is very important so GPUs prefer “structs of arrays” to “arrays of structures”. As you did moving field “blocked” into vector “obstacle”, it should be advantageous to convert all of the fields of “grid_cell” into separate vectors. This should show benefit on CPU as well for loops that don’t access all of the fields.

2. If “obstacle” is very sparse (which I guess is unlikely) then moving it to its own vector is particularly value. GPUs like CPUs will load more than one word from the memory system either in cache lines or some other form and you waste bandwidth when you don’t need some of the data. For many system memory bandwidth is the bottleneck resource so any way to reduce bandwidth helps.

3. This is more speculative, but now that you are writing all of the output vector, it is possible the memory subsystem is avoiding reading values in “node” that will simply be overwritten

4. On some systems, the on-chip memory is split into banks and having an odd number of fields within your structure may help remove bank conflicts.

5. Some systems will also “vectorize” loads and stores so again removing “blocked” from the structure might enable more vectorization. The shift to struct-of-arrays mitigates this worry.

Thanks for your interest in C++ AMP.

David Callahan

http://blogs.msdn.com/b/nativeconcurrency/ C++ AMP Team Blog