Does __syncthreads() synchronize all threads in the grid?

Question

...or just the threads in the current warp or block?

Also, when the threads in a particular block encounter (in the kernel) the following line

__shared__         


        

Answer 1

In order to provide further details, aside of the answers, quoting seibert:

More generally, __syncthreads() is a barrier primitive designed to protect you from read-after-write memory race conditions within a block.

The rules of use are pretty simple:

1. Put a __syncthreads() after the write and before the read when there is a possibility of a thread reading a memory location that another thread has written to.

2. __syncthreads() is only a barrier within a block, so it cannot protect you from read-after-write race conditions in global memory unless the only possible conflict is between threads in the same block. __syncthreads() is pretty much always used to protect shared memory read-after-write.

3. Do not use a __syncthreads() call in a branch or a loop until you are sure every single thread will reach the same __syncthreads() call. This can sometimes require that you break your if-blocks into several pieces to put __syncthread() calls at the top-level where all threads (including those which failed the if predicate) will execute them.

4. When looking for read-after-write situations in loops, it helps to unroll the loop in your head when figuring out where to put __syncthread() calls. For example, you often need an extra __syncthreads() call at the end of the loop if there are reads and writes from different threads to the same shared memory location in the loop.

5. __syncthreads() does not mark a critical section, so don’t use it like that.

6. Do not put a __syncthreads() at the end of a kernel call. There’s no need for it.

7. Many kernels do not need __syncthreads() at all because two different threads never access the same memory location.