Random Number Streams on a GPU

By default, the random number generation functions rand, randi, and randn use different generator settings for calculations on a GPU compared to those on a CPU. You can change the behavior of random number generators to generate reproducible sequences of random numbers on the GPU and CPU.

The table below summarizes the default settings for the GPU and CPU on client and worker MATLAB® sessions:

 GeneratorSeedNormal Transform
Client CPU'Twister' or 'mt19937ar'0'Ziggurat'
Worker CPU'Threefry' or 'Threefry4x64_20'0'Inversion'
GPU (on client or worker)'Threefry' or 'Threefry4x64_20'0'BoxMuller'

In most cases, it does not matter that the default random number generator on the GPU is not the same as the default generators on the client or worker CPU. However, if you need to reproduce the same results on both the GPU and CPU, you can set the generators accordingly.

Client CPU and GPU

In a fresh MATLAB session, MATLAB generates different sequences of random numbers on the CPU and GPU.

Rc = rand(1,4)
Rc =
    0.8147    0.9058    0.1270    0.9134
Rg = rand(1,4,'gpuArray')
Rg =
    0.3640    0.5421    0.6543    0.7436

If you need to generate the same sequence of random numbers on both the GPU and CPU, you can set the generator settings to match.

There are three random number generator algorithms available on the GPU: 'Threefry', 'Philox', and 'CombRecursive'. All are supported on the CPU. The following table lists the algorithms for these generators and their properties.

KeywordGeneratorMultiple Stream and Substream SupportApproximate Period in Full Precision
'Threefry' or 'Threefry4x64_20'Threefry 4x64 generator with 20 roundsYes2514 (2256 streams of length 2258)
'Philox' or 'Philox4x32_10'Philox 4x32 generator with 10 roundsYes2193 (264 streams of length 2129)
'CombRecursive' or 'mrg32k3a'Combined multiple recursive generatorYes2191 (263 streams of length 2127)

You can use rng and gpurng to set the generator algorithm and seed on the CPU and GPU, respectively.

sc = rng(1,'Threefry');
Rc = rand(1,4)
Rc =
   0.1404    0.8197    0.1073    0.4131
sg = gpurng(1,'Threefry');
Rg = rand(1,4,'gpuArray')
Rg =
    0.1404    0.8197    0.1073    0.4131

rand and randi now generate the same sequences of random numbers on the client CPU and GPU.

Worker CPU and GPU

A parallel worker CPU uses the same default random number generator type and seed as the client GPU and the worker GPU, if it has one. The GPU and CPU do not share the same stream. By default, rand and randi generate the same sequence of numbers on a GPU and a worker CPU.

The settings are different from those on the client CPU. For more information, see Control Random Number Streams on Workers

If you need to generate different random numbers on each worker, you can change the generator settings. In this example, each worker creates the same sequence on its GPU and CPU, but different sequences are generated on each worker.

p = parpool(2);
spmd
    rng(labindex,'Threefry');
    Rc = rand(1,4)

    gpurng(labindex,'Threefry');
    Rg = rand(1,4,'gpuArray')
end
delete(p)

Normally Distributed Numbers

For normally distributed random numbers created using the randn function, MATLAB produces different results on a client CPU, a worker CPU and a GPU. The transformation of uniform random numbers into normally distributed random numbers is controlled by the NormalTransform setting. You can control this on the GPU using parallel.gpu.RandStream.

On a client CPU, the default 'NormalTransform' setting is 'Ziggurat'. On a worker CPU, the default setting is 'Inversion'.

Unless otherwise specified, GPU code uses the 'BoxMuller' transform for the 'Threefry' and 'Philox' generators and the 'Inversion' transform for the 'CombRecursive' generator.

You can set the same generators and transforms on the CPU and the GPU to get the same randn sequences. The only transform supported on both the CPU and GPU is the 'Inversion' transform.

sc = RandStream('Threefry','NormalTransform','Inversion','Seed',1);
RandStream.setGlobalStream(sc)

sg = parallel.gpu.RandStream('Threefry','NormalTransform','Inversion','Seed',1);
parallel.gpu.RandStream.setGlobalStream(sg);

Rc = randn(1,4)
Rc =
   -1.0783    0.9144   -1.2412   -0.2196
Rg = randn(1,4,'gpuArray')
Rg =
   -1.0783    0.9144   -1.2412   -0.2196

See Also

| | |

Related Topics