"Maximum variable size allowed on the device is exceeded." when using predict on GPU

61 vues (au cours des 30 derniers jours)
hanspeter
hanspeter le 30 Oct 2024 à 9:58
Commenté : hanspeter le 3 Nov 2024 à 13:17
Hello there,
I have a neural net and want to do a prediction using predict. When I do that I get the error
Error using dlnetwork/predict
Execution failed during layer(s) 'fc1'.
Caused by:
Error using parallel.internal.gpu.mtimesAdd
Maximum variable size allowed on the device is exceeded.
Afaik it's because on GPU Matlab only supports 2^32 elements in one array. Whats the best way to circumvent this?
I know I can disable the GPU completely by setting CUDA_VISIBLE_DEVICES to -1, but I still want to use it in other functions.
The data I pass into predict is a normal array and not a gpuarray.
Is there a way to force predict to use CPU?
Cheers

Réponse acceptée

Joss Knight
Joss Knight le 1 Nov 2024 à 15:39
Modifié(e) : Joss Knight le 1 Nov 2024 à 15:41
Is there a way to force predict to use CPU?
Yes. Pass data in as an in-memory array. If your data is on the device, you may need to call gather.
z = predict(net, gather(x));
If this does not work it will be because some of your network weights are on the GPU. So you may need to call gather on the network as well:
net = dlupdate(@gather, net);
net.State = dlupdate(@gather, net.State);
It's much more likely, however, that you're just passing too much data into your network. If you are trying to call predict on a large dataset, consider using minibatchpredict instead, to process the data in batches.
  1 commentaire
hanspeter
hanspeter le 3 Nov 2024 à 13:17
Thanks, that's it. I already used gather(x), but didn't think of also gathering the weights

Connectez-vous pour commenter.

Plus de réponses (1)

Shivam Lahoti
Shivam Lahoti le 30 Oct 2024 à 11:22
Hi hanspeter,
I understand you are encountering a “Maximum variable size allowed on the device is exceeded" error when using predict on a GPU.
As you mentioned, the current maximum number of elements allowed in a gpuArray is intmax('int32'), or 2^31. This is because CUBLAS passes array lengths as int (int32_T). This is a known limitation and is mentioned in the documentation link below:
Moreover, none of the following can exceed intmax('int32'):
  • The number of elements of a dense array.
  • The number of nonzero elements of a sparse array.
  • The size in any given dimension. For example, zeros(0,3e9,'gpuArray') is not allowed.
I hope this helps you understand the issue.
Regards,
Shivam

Catégories

En savoir plus sur Image Data Workflows dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by