CUDA's Mersenne Twister for an arbitrary number of threads

Question

CUDA's implementation of the Mersenne Twister (MT) random number generator is limited to a maximal number of threads/blocks of 256 and 200 blocks/grid, i.e. the maximal number of threads is 51200.

Therefore, it is not possible to launch the kernel that uses the MT with

kernel<<<blocksPerGrid, threadsPerBlock>>>(devMTGPStates, ...)

where

int blocksPerGrid = (n+threadsPerBlock-1)/threadsPerBlock;

and n is the total number of threads.

What is the best way to use the MT for threads > 51200?

My approach if to use constant values for blocksPerGrid and threadsPerBlock, e.g. <<<128,128>>> and use the following in the kernel code:

__global__ void kernel(curandStateMtgp32 *state, int n, ...) { 

    int id = threadIdx.x+blockIdx.x*blockDim.x;

    while (id < n) {

        float x = curand_normal(&state[blockIdx.x]);
        /* some more calls to curand_normal() followed
           by the algorithm that works with the data */

        id += blockDim.x*gridDim.x; 
    }
}

I am not sure if this is the correct way or if it can influence the MT status in an undesired way?

Thank you.

Answer 1

I suggest you read the CURAND documentation carefully and thoroughly.

The MT API will be most efficient when using 256 threads per block with up to 64 blocks to generate numbers.

If you need more than that, you have a variety of options:

1. simply generate more numbers from the existing state - set (i.e. 64 blocks, 256 threads), and distribute these numbers amongst the threads that need them.
2. Use more than a single state per block (but this does not allow you to exceed the overall limit within a state-set, it just addresses the need for a single block.)
3. Create multiple MT generators with independent seeds (and therefore independent state-sets).

Generally, I don't see a problem with the kernel that you've outlined, and it's roughly in line with choice 1 above. However it does not allow you to exceed 51200 threads. (your example has <<<128, 128>>> so 16384 threads)

Answer 2

Following Robert's answer, below I'm providing a fully worked example on using cuRAND's Mersenne Twister for an arbitrary number of threads. I'm using Robert's first option to generate more numbers from the existing state-set and distributing these numbers amongst the threads that need them.

// --- Generate random numbers with cuRAND's Mersenne Twister

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

#include <cuda.h>
#include <curand_kernel.h>
/* include MTGP host helper functions */
#include <curand_mtgp32_host.h>

#define BLOCKSIZE   256
#define GRIDSIZE    64

/*******************/
/* GPU ERROR CHECK */
/*******************/
#define gpuErrchk(x) do { if((x) != cudaSuccess) { \
    printf("Error at %s:%d\n",__FILE__,__LINE__); \
    return EXIT_FAILURE;}} while(0)

#define CURAND_CALL(x) do { if((x) != CURAND_STATUS_SUCCESS) { \
    printf("Error at %s:%d\n",__FILE__,__LINE__); \
    return EXIT_FAILURE;}} while(0)

/*******************/
/* iDivUp FUNCTION */
/*******************/
__host__ __device__ int iDivUp(int a, int b) { return ((a % b) != 0) ? (a / b + 1) : (a / b); }

/*********************/
/* GENERATION KERNEL */
/*********************/
__global__ void generate_kernel(curandStateMtgp32 * __restrict__ state, float * __restrict__ result, const int N)
{
    int tid = threadIdx.x + blockIdx.x * blockDim.x;
    for (int k = tid; k < N; k += blockDim.x * gridDim.x)
        result[k] = curand_uniform(&state[blockIdx.x]);
}

/********/
/* MAIN */
/********/
int main()
{
    const int N = 217 * 123;

    // --- Allocate space for results on host
    float *hostResults = (float *)malloc(N * sizeof(float));

    // --- Allocate and initialize space for results on device 
    float *devResults; gpuErrchk(cudaMalloc(&devResults, N * sizeof(float)));
    gpuErrchk(cudaMemset(devResults, 0, N * sizeof(float)));

    // --- Setup the pseudorandom number generator
    curandStateMtgp32 *devMTGPStates; gpuErrchk(cudaMalloc(&devMTGPStates, GRIDSIZE * sizeof(curandStateMtgp32)));
    mtgp32_kernel_params *devKernelParams; gpuErrchk(cudaMalloc(&devKernelParams, sizeof(mtgp32_kernel_params)));
    CURAND_CALL(curandMakeMTGP32Constants(mtgp32dc_params_fast_11213, devKernelParams));
    //CURAND_CALL(curandMakeMTGP32KernelState(devMTGPStates, mtgp32dc_params_fast_11213, devKernelParams, GRIDSIZE, 1234));
    CURAND_CALL(curandMakeMTGP32KernelState(devMTGPStates, mtgp32dc_params_fast_11213, devKernelParams, GRIDSIZE, time(NULL)));

    // --- Generate pseudo-random sequence and copy to the host
    generate_kernel << <GRIDSIZE, BLOCKSIZE >> >(devMTGPStates, devResults, N);
    gpuErrchk(cudaPeekAtLastError());
    gpuErrchk(cudaDeviceSynchronize());
    gpuErrchk(cudaMemcpy(hostResults, devResults, N * sizeof(float), cudaMemcpyDeviceToHost));

    // --- Print results
    //for (int i = 0; i < N; i++) {
    for (int i = 0; i < 10; i++) {
        printf("%f\n", hostResults[i]);
    }

    // --- Cleanup
    gpuErrchk(cudaFree(devMTGPStates));
    gpuErrchk(cudaFree(devResults));
    free(hostResults);

    return 0;
}