GPU Array lazy evaluation?

I have found this behavior of the MATLAB gpuArray class by chance, as a consequence of several unexpected out of memory error on some code which is of none importance here. Can anybody explain this? An array, say 40000*5000 in single precision should take 0.8 GB. If I run the following operations in sequence

H=zeros([40000*5000 1],'single','gpuArray');
H=H+randn([40000*5000 1],'single','gpuArray');
H=H+randn([40000*5000 1],'single','gpuArray');
H=zeros([40000*5000 1],'single','gpuArray');

the GPU RAM shows 3297MB occupied, of which 200 are taken by default by MATLAB when the GPU is selected. However, if I clear the variable H the memory is freed totally, which means that is not a leak. It seems that MATLAB for certain operations reserves some memory or even duplicates the target matrix. Furthermore, if I check the available memory (not the free memory) it shows the expected matrix size being taken. However, the "locked" memory is unavailable, because I have been monitoring my computations and the loop just crashes when the max mem size is reached.

Can anybody, please, explain this behavior and if a workaround exists, still using the gpuArray?

I am running a NVIDIA GTX 1080ti with XEON E5-2860 v2 on ubuntu 16.04, with MATLAB2018a

[fpexp]

I have found this example related to the behavior of the ArrayFire library. It should be something called "lazy evaluation"? (pardon the lack of knowledge). Is this what MATLAB does? on which CUDA libraries the GPU Array functions relay?

p.s. forgot to say that the CUDA version is the 9.1, however this thing happens with 9.0 as well and with both toolkit also on MATLAB2017b.

Answer 1

[Joss Knight]

MATLAB doesn't take 200MB on device selection, the CUDA driver does.

MATLAB pools up to a quarter of the GPU memory by default. This considerably improves performance by reducing the number of synchronous allocations.

You can turn off pooling using

feature('GpuAllocPoolSizeKb', 0)