MATLAB Answers

Can't load Python module on Matlab production server

9 views (last 30 days)
Marc Youcef
Marc Youcef on 3 Apr 2020
Answered: Marc Youcef on 15 Oct 2020
I am trying to load a user defined Python module on Matlab Production Server but it is not working. It is working on my localt computer. On both local and server I have Python 3.7 installed through Anaconda Installation. I am running under Windows 7 for my local and Win10 on the server.
On the server, I called a python script executing the loading of this module and it is working. I also did within the function compiled on the matlab server :
system('python mypythonscript.py')
This worked as well.
Both local and server have their pyenv well defined.
This is the script not working on Production Server :
%% Setting Python Env
modpath = 'D:\PythonLibs\my_module';
P = py.sys.path
if count(P,modpath) == 0
insert(P,int32(0),modpath);
end
system('python mypythonscript.py')
%% Loading Preform Library
mod = py.importlib.import_module('my_module');
This is what I got from the server:
375 [2020.04.03 16:58:34.878276] [out] [worker:1] PyVersion =
376 [2020.04.03 16:58:34.878276] [out] [worker:1]
377 [2020.04.03 16:58:34.878276] [out] [worker:1] PythonEnvironment with properties:
378 [2020.04.03 16:58:34.878276] [out] [worker:1]
379 [2020.04.03 16:58:34.878276] [out] [worker:1] Version: "3.7"
380 [2020.04.03 16:58:34.878276] [out] [worker:1] Executable: "D:\py37\python.EXE"
381 [2020.04.03 16:58:34.878276] [out] [worker:1] Library: "D:\py37\python37.dll"
382 [2020.04.03 16:58:34.878276] [out] [worker:1] Home: "D:\py37"
383 [2020.04.03 16:58:34.878276] [out] [worker:1] Status: Loaded
384 [2020.04.03 16:58:34.878276] [out] [worker:1] ExecutionMode: InProcess
385 [2020.04.03 16:58:34.878276] [out] [worker:1] ProcessID: "5800"
386 [2020.04.03 16:58:34.878276] [out] [worker:1] ProcessName: "MATLAB"
387 [2020.04.03 16:58:34.878276] [out] [worker:1]
388 [2020.04.03 16:58:34.878276] [out] [worker:1]
389 [2020.04.03 16:58:34.878276] [out] [worker:1] ans =
390 [2020.04.03 16:58:34.878276] [out] [worker:1]
391 [2020.04.03 16:58:34.878276] [out] [worker:1] 99
392 [2020.04.03 16:58:34.878276] [out] [worker:1]
393 [2020.04.03 16:58:36.222065] [out] [worker:1]
394 [2020.04.03 16:58:36.222065] [out] [worker:1] P =
395 [2020.04.03 16:58:36.222065] [out] [worker:1]
396 [2020.04.03 16:58:36.222065] [out] [worker:1] Python list with no properties.
397 [2020.04.03 16:58:36.222065] [out] [worker:1]
398 [2020.04.03 16:58:36.222065] [out] [worker:1] ['D:\\PythonLibs\\my_module', '', 'D:\\py37\\python37.zip', 'D:\\py37\\DLLs', 'D:\\py37\\lib', 'D:\\py37', 'D:\\py37\\lib\\site-packages', 'D:\\py37\\lib\\site-packages\\win32', 'D:\\py37\\lib\\site-packages\\win32\\lib', 'D:\\py37\\lib\\site-packages\\Pythonwin']
399 [2020.04.03 16:58:36.222065] [out] [worker:1]
400 [2020.04.03 16:58:36.222065] [out] [worker:1]
401 [2020.04.03 16:58:36.222065] [out] [worker:1] ans =
402 [2020.04.03 16:58:36.222065] [out] [worker:1]
403 [2020.04.03 16:58:36.222065] [out] [worker:1] 0
404 [2020.04.03 16:58:36.222065] [out] [worker:1]
405 [2020.04.03 16:58:36.222065] [err] [worker:1] Error using <frozen importlib>_find_and_load_unlocked (line 965)
406 [2020.04.03 16:58:36.222065] [err] [worker:1] Python Error: ModuleNotFoundError: No module named 'my_module'
407 [2020.04.03 16:58:36.222065] [err] [worker:1]
408 [2020.04.03 16:58:36.222065] [err] [worker:1] Error in <frozen importlib>_find_and_load (line 983)
409 [2020.04.03 16:58:36.222065] [err] [worker:1]
410 [2020.04.03 16:58:36.222065] [err] [worker:1] Error in <frozen importlib>_gcd_import (line 1006)
411 [2020.04.03 16:58:36.222065] [err] [worker:1]
412 [2020.04.03 16:58:36.222065] [err] [worker:1] Error in __init__>import_module (line 127)
The ans = 0 is a response from the system command running the python script which load the user defined module. So it shows that it works when executed from python itself. I also tested to run basic python function on MPS (Matlab Production Server) like py.abs(-99) and it worked. So everything looks fine on the Python installation and its usage from MPS except loading that user defined module.
I have tried adding the module in the archive before compiling it, and accessing it through ctfroot - Did not work.
Tried to set it in different folder on my D drive and giving absolute path - Did not work.
Again, everything is working fine locally and tested the Test Server from MPS Compiler and works.
Only when deplying to server that I have this failure. I have exhausted during hours options I had in mind and on forums.
Thanks for any support.

More Answers (1)

Kojiro Saito
Kojiro Saito on 26 May 2020
I suspect that Python in the server might be mixed with 32bit and 64bit. Could you confirm you're using only 64 bit version of Python?
  1 Comment
Marc Youcef
Marc Youcef on 15 Oct 2020
That was not the issue. Both library parent folder and library path need to be added to sys.path.
However, I found no means on modifying the python environment on matlab production server. Each time MPS says that python is loaded and I can't modify env, meaning I need to restart. But I am not on Matlab desktop to restart. Restarting the instance did not solve the issue and I am not using any py.xxx command before trying to set a specific env to python using pyenv.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by