Can't load Python module on Matlab production server

I am trying to load a user defined Python module on Matlab Production Server but it is not working. It is working on my localt computer. On both local and server I have Python 3.7 installed through Anaconda Installation. I am running under Windows 7 for my local and Win10 on the server.

On the server, I called a python script executing the loading of this module and it is working. I also did within the function compiled on the matlab server :

system('python mypythonscript.py')

This worked as well.

Both local and server have their pyenv well defined.

This is the script not working on Production Server :

%% Setting Python Env
modpath = 'D:\PythonLibs\my_module';

P = py.sys.path
if count(P,modpath) == 0
    insert(P,int32(0),modpath);
end
system('python mypythonscript.py')
%% Loading Preform Library
mod = py.importlib.import_module('my_module');

This is what I got from the server:

375 [2020.04.03 16:58:34.878276] [out] [worker:1] PyVersion =

376 [2020.04.03 16:58:34.878276] [out] [worker:1]

377 [2020.04.03 16:58:34.878276] [out] [worker:1] PythonEnvironment with properties:

378 [2020.04.03 16:58:34.878276] [out] [worker:1]

379 [2020.04.03 16:58:34.878276] [out] [worker:1] Version: "3.7"

380 [2020.04.03 16:58:34.878276] [out] [worker:1] Executable: "D:\py37\python.EXE"

381 [2020.04.03 16:58:34.878276] [out] [worker:1] Library: "D:\py37\python37.dll"

382 [2020.04.03 16:58:34.878276] [out] [worker:1] Home: "D:\py37"

383 [2020.04.03 16:58:34.878276] [out] [worker:1] Status: Loaded

384 [2020.04.03 16:58:34.878276] [out] [worker:1] ExecutionMode: InProcess

385 [2020.04.03 16:58:34.878276] [out] [worker:1] ProcessID: "5800"

386 [2020.04.03 16:58:34.878276] [out] [worker:1] ProcessName: "MATLAB"

387 [2020.04.03 16:58:34.878276] [out] [worker:1]

388 [2020.04.03 16:58:34.878276] [out] [worker:1]

389 [2020.04.03 16:58:34.878276] [out] [worker:1] ans =

390 [2020.04.03 16:58:34.878276] [out] [worker:1]

391 [2020.04.03 16:58:34.878276] [out] [worker:1] 99

392 [2020.04.03 16:58:34.878276] [out] [worker:1]

393 [2020.04.03 16:58:36.222065] [out] [worker:1]

394 [2020.04.03 16:58:36.222065] [out] [worker:1] P =

395 [2020.04.03 16:58:36.222065] [out] [worker:1]

396 [2020.04.03 16:58:36.222065] [out] [worker:1] Python list with no properties.

397 [2020.04.03 16:58:36.222065] [out] [worker:1]

398 [2020.04.03 16:58:36.222065] [out] [worker:1] ['D:\\PythonLibs\\my_module', '', 'D:\\py37\\python37.zip', 'D:\\py37\\DLLs', 'D:\\py37\\lib', 'D:\\py37', 'D:\\py37\\lib\\site-packages', 'D:\\py37\\lib\\site-packages\\win32', 'D:\\py37\\lib\\site-packages\\win32\\lib', 'D:\\py37\\lib\\site-packages\\Pythonwin']

399 [2020.04.03 16:58:36.222065] [out] [worker:1]

400 [2020.04.03 16:58:36.222065] [out] [worker:1]

401 [2020.04.03 16:58:36.222065] [out] [worker:1] ans =

402 [2020.04.03 16:58:36.222065] [out] [worker:1]

403 [2020.04.03 16:58:36.222065] [out] [worker:1] 0

404 [2020.04.03 16:58:36.222065] [out] [worker:1]

405 [2020.04.03 16:58:36.222065] [err] [worker:1] Error using <frozen importlib>_find_and_load_unlocked (line 965)

406 [2020.04.03 16:58:36.222065] [err] [worker:1] Python Error: ModuleNotFoundError: No module named 'my_module'

407 [2020.04.03 16:58:36.222065] [err] [worker:1]

408 [2020.04.03 16:58:36.222065] [err] [worker:1] Error in <frozen importlib>_find_and_load (line 983)

409 [2020.04.03 16:58:36.222065] [err] [worker:1]

410 [2020.04.03 16:58:36.222065] [err] [worker:1] Error in <frozen importlib>_gcd_import (line 1006)

411 [2020.04.03 16:58:36.222065] [err] [worker:1]

412 [2020.04.03 16:58:36.222065] [err] [worker:1] Error in __init__>import_module (line 127)

The ans = 0 is a response from the system command running the python script which load the user defined module. So it shows that it works when executed from python itself. I also tested to run basic python function on MPS (Matlab Production Server) like py.abs(-99) and it worked. So everything looks fine on the Python installation and its usage from MPS except loading that user defined module.

I have tried adding the module in the archive before compiling it, and accessing it through ctfroot - Did not work.

Tried to set it in different folder on my D drive and giving absolute path - Did not work.

Again, everything is working fine locally and tested the Test Server from MPS Compiler and works.

Only when deplying to server that I have this failure. I have exhausted during hours options I had in mind and on forums.

Thanks for any support.

Answer 1

[Marc Youcef]

Refer to my comment above.

Answer 2

[Kojiro Saito]

I suspect that Python in the server might be mixed with 32bit and 64bit. Could you confirm you're using only 64 bit version of Python?

https://www.mathworks.com/help/matlab/matlab_external/system-and-configuration-requirements.html

[Marc Youcef]

That was not the issue. Both library parent folder and library path need to be added to sys.path.

However, I found no means on modifying the python environment on matlab production server. Each time MPS says that python is loaded and I can't modify env, meaning I need to restart. But I am not on Matlab desktop to restart. Restarting the instance did not solve the issue and I am not using any py.xxx command before trying to set a specific env to python using pyenv.