Python + MATLAB Integration Output Issue

Question/background:

• Has anyone encountered an issue with MATLAB where it does not write to the results/output .mat files when executed through Python's MATLAB Engine API on a virtual desktop?
• I have confirmed that it works on my Windows laptop, but now that I am trying to get it to run on an Amazon Workspaces VDI, the models run all the way through but the output files are empty.
• When I run the models only through the MATLAB console, it will write to the output files as expected. It is only when running the "run models" function through python MATLAB engine that it does not write the outputs out.
• There are no error messages being output by MATLAB related to not being able to update the results files
• As mentioned below, most of the MATLAB model code is obscured by .p files

Technologies Used/Environment:

• Python GUI built with PyQT that executes MATLAB/Simulink Models through MATLAB Engine API for Python
• Python 3.11 + MATLAB 2023b
• Amazon Workspaces VDI

Checks I have done so far

• The whole pipeline has been tested with my Windows laptop
• On the VDI:
• Confirmed that I can write a file with MATLAB into the directory that the output files live when executing with Python MATLAB engine
• Confirmed that MATLAB and Python have the same access permissions/user roles
• Confirmed that all file paths that MATLAB and Python-Executed MATLAB are executing (working directories) are identical

I don't have much MATLAB background, so any help would be greatly appreciated!

[Torsten]

Your question is very specific - you should contact MATLAB support:

https://uk.mathworks.com/support/contact_us.html

Here are three possible causes identified by AI:

1. The Buffer and Synchronization Issue (Most Likely Root Cause)

When you call MATLAB via the Python engine, MATLAB runs as a headless background process (often without a GUI thread). When the models finish running, MATLAB might only write the data to the RAM cache. If the Python process abruptly closes the engine or the script ends before this cache is flushed to the VDI's hard drive, the .mat file remains physically empty (0 KB). VDIs often use optimized, network-based file systems (such as AWS FSx or EBS) that cache data much more aggressively than local SSDs.

• Solution: Force MATLAB to close all file handles and flush the cache immediately after the model execution in your Python code.
• Action: Add the following engine calls to your Python script directly after the command that executes the models:

python

# Your model execution (example)
eng.run_models(nargout=0) 

# Force all cached data to be written to the disk
eng.eval("clear all;", nargout=0) # Clears the workspace and forces file closes
eng.eval("pack;", nargout=0)      # Consolidates memory (optional)

2. Missing Environment Variables in the Engine Process

When the Python engine starts MATLAB, MATLAB inherits the environment variables of the Python process. On Amazon WorkSpaces, user-specific paths (like AppData\Local\Temp or network drives) are often managed via system variables. If your obscured .p files internally rely on relative paths or temporary directories to generate the final data, this will fail because the engine process cannot resolve these paths. MATLAB often does not throw an error in this scenario; it simply aborts the write process silently.

• Solution: Verify if MATLAB sees the same system paths when running under Python.
• Action: Print the MATLAB environment via Python and compare it with the standalone MATLAB console:

python

print(eng.eval("getenv('TEMP')"))
print(eng.eval("pwd"))

If you spot differences compared to the working standalone MATLAB console, you must explicitly set the paths in Python before starting the model:

python

eng.eval("cd('C:\\Your\\Exact\\Path')", nargout=0)

3. Simulink Process Priority & Thread Blocking in AWS WorkSpaces

Since you mentioned that Simulink models are involved: Simulink often spawns its own worker threads or background processes for simulations. On a VDI (where CPU cores are virtualized and often strictly limited or shared), the MATLAB parent process started by Python might terminate before the asynchronous Simulink write process has finished. Because the .p files obscure the code, you cannot see whether the write command is triggered asynchronously at the very end of the simulation.

• Solution: Actively block the Python process until Simulink has definitively shut down completely.
• Action: Use an explicit bdclose (Bundle Close) for Simulink in Python:

python

eng.run_models(nargout=0)
# Explicitly close all Simulink models and force the simulation to terminate
eng.eval("bdclose('all');", nargout=0) 

Summary for Your Python Code

Try protecting the engine teardown sequence like this:

python

# 1. Run the models
eng.run_models(nargout=0)

# 2. Explicitly close Simulink (in case the .p files do not clean up properly)
eng.eval("bdclose('all');", nargout=0)

# 3. Clear the workspace to release file handles
eng.eval("clear all;", nargout=0)

# 4. Safely terminate the engine
eng.quit()