Random Number Streams on a GPU
By default, the random number generation functions
randn use different generator settings for calculations on a GPU
compared to those on a CPU. You can change the behavior of random number generators to
generate reproducible sequences of random numbers on the GPU and CPU.
The table below summarizes the default settings for the GPU and CPU on client and worker MATLAB® sessions:
|GPU (on client or worker)|
In most cases, it does not matter that the default random number generator on the GPU is not the same as the default generators on the client or worker CPU. However, if you need to reproduce the same results on both the GPU and CPU, you can set the generators accordingly.
Client CPU and GPU
In a fresh MATLAB session, MATLAB generates different sequences of random numbers on the CPU and GPU.
Rc = rand(1,4)
Rc = 0.8147 0.9058 0.1270 0.9134
Rg = rand(1,4,'gpuArray')
Rg = 0.3640 0.5421 0.6543 0.7436
If you need to generate the same sequence of random numbers on both the GPU and CPU, you can set the generator settings to match.
There are three random number generator algorithms available on the GPU:
'CombRecursive'. All are supported on the CPU. The following
table lists the algorithms for these generators and their properties.
|Keyword||Generator||Multiple Stream and Substream Support||Approximate Period in Full Precision|
|Threefry 4x64 generator with 20 rounds||Yes||2514 (2256 streams of length 2258)|
|Philox 4x32 generator with 10 rounds||Yes||2193 (264 streams of length 2129)|
|Combined multiple recursive generator||Yes||2191 (263 streams of length 2127)|
sc = rng(1,'Threefry'); Rc = rand(1,4)
Rc = 0.1404 0.8197 0.1073 0.4131
sg = gpurng(1,'Threefry'); Rg = rand(1,4,'gpuArray')
Rg = 0.1404 0.8197 0.1073 0.4131
randi now generate the same
sequences of random numbers on the client CPU and GPU.
Worker CPU and GPU
A parallel worker CPU uses the same default random number generator type and seed
as the client GPU and the worker GPU, if it has one. The GPU and CPU do not share
the same stream. By default,
generate the same sequence of numbers on a GPU and a worker CPU.
The settings are different from those on the client CPU. For more information, see Control Random Number Streams on Workers
If you need to generate different random numbers on each worker, you can change the generator settings. In this example, each worker creates the same sequence on its GPU and CPU, but different sequences are generated on each worker.
p = parpool(2); spmd rng(labindex,'Threefry'); Rc = rand(1,4) gpurng(labindex,'Threefry'); Rg = rand(1,4,'gpuArray') end delete(p)
Normally Distributed Random Numbers
For normally distributed random numbers created using the
randn function, MATLAB produces different results on a client CPU, a worker CPU and a GPU.
The transformation of uniform random numbers into normally distributed random
numbers is controlled by the
NormalTransform setting. You can
control this on the GPU using
On a client CPU, the default
'NormalTransform' setting is
'Ziggurat'. On a worker CPU, the default setting is
Unless otherwise specified, GPU code uses the
transform for the
generators and the
'Inversion' transform for the
You can set the same generators and transforms on the CPU and the GPU to get the
randn sequences. The only transform supported on both the
CPU and GPU is the
sc = RandStream('Threefry','NormalTransform','Inversion','Seed',1); RandStream.setGlobalStream(sc) sg = parallel.gpu.RandStream('Threefry','NormalTransform','Inversion','Seed',1); parallel.gpu.RandStream.setGlobalStream(sg); Rc = randn(1,4)
Rc = -1.0783 0.9144 -1.2412 -0.2196
Rg = randn(1,4,'gpuArray')
Rg = -1.0783 0.9144 -1.2412 -0.2196