Why am I unable to load a Python module that requires a certain CUDA version in MATLAB after calling a MATLAB function that uses CUDA enabled GPU?

I am using the Deep Learning Toolbox and GPU computation in MATLAB R2021b with the function "predict".
After calling this function, I wish to load the Python module PyTorch into MATLAB. My PyTorch version is 2.1.2+cu121, which means it comes with and uses CUDA version 12.1.
However, when I run the command to import PyTorch in MATLAB, the following error is thrown.

Error using __init__><module> (line 130)
Python Error: OSError: [WinError 127] The specified procedure could not be
found. Error loading "D:\Program
Files\Python\Python39\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll" or
one of its dependencies.
Error in <frozen importlib>_call_with_frames_removed (line 228)
Error in <frozen importlib>exec_module (line 850)
Error in <frozen importlib>_load_unlocked (line 680)
Error in <frozen importlib>_find_and_load_unlocked (line 986)
Error in <frozen importlib>_find_and_load (line 1007)
Error in <frozen importlib>_gcd_import (line 1030)
Error in __init__>import_module (line 127)
Why is this error happening and how can I load PyTorch?
Additionally, if I try to load PyTorch before calling "predict", the output of "predict" is different than the output is if I never load "PyTorch". This difference in output is also a concern for me.

Answer 1

[MathWorks Support Team]

In order to work with any Python library that requires a different version of a 3rd party library also required by MATLAB, the Python environment must be run in "OutOfProcess" execution mode. This applies to the 3rd party CUDA library that is used by MATLAB (with "predict") and PyTorch.

 

Since MATLAB R2021b uses CUDA version 11.0 and the PyTorch version that you are using has CUDA version 12.1, there is a mismatch with the CUDA library versions. This mismatch is what causes both the error and the differing output when using "InProcess" execution mode, which is the default for the Python environment in MATLAB.

 

To switch to "OutOfProcess" execution mode, run the following command before calling any "py" commands in MATLAB.

>> pyenv("ExecutionMode", "OutOfProcess")

Afterwards, you should be able to load PyTorch and use "predict" as intended.

See the following documentation for more details.

https://www.mathworks.com/help/releases/R2021b/matlab/ref/pyenv.html#mw_678123e1-e5e5-46bc-bce5-f5e769f5a684

If you are concerned about the performance of "OutOfProcess" execution mode, then the recommend workaround is to ensure that the CUDA version of any imported module matches the CUDA version supported by your MATLAB release version. 

In the case of using MATLAB R2021b which uses CUDA version 11.0, I would recommend installing and loading PyTorch 1.7.1+cu110 with CUDA version 11.0.

 

Note that all MATLAB release versions ship with a certain CUDA version. See the following link to determine which CUDA version you should use depending on your MATLAB release.

https://www.mathworks.com/help/parallel-computing/run-mex-functions-containing-cuda-code.html#mw_20acaa78-994d-4695-ab4b-bca1cfc3dbac