Is there a way to remove or rename a variable in a big MAT file?

I want to rename/remove a variable in very large MAT file but want to avoid load all the data, modify variables and write all new data to the disk.

The MAT file is hundreds GB and load/write all leads to a heavy IO load.

[Jan]

Sorry, Ray Lee, my mistake. I've removed my comment.

If you want to remove a variable, it must be deleted from the hard disk. Because the -v7.3 format compress the data, the data must be uncompressed at first, then removed, and finally the rest of the file must be shifted to release the memory on the disk. I do not expect, that this can be done without a heavy IO load. If you want to release the memory occupied by a variable, you have to move the memory taken by the other variables, maybe including some de- and re-compressing.

So my opinion is, that a MAT file with several hundreds GB is a bad choice to store the data.

[Stephen23]

"I want to rename..."

Why do the variables need to be renamed? This is not an obtuse question, but if you are putting some meta-data into variable names then it is quite possible that the entire data-design could be improved.

[Ray Lee]

For historical reasons, variables were renamed in different convention.

And also, in some cases, the MAT files were saved with typos in varnames. I met such situations more than once.

[Stephen23]

@Ray Lee: fair enough.

As an alternative it may be possible to process the fieldnames using some regular expressions (or something similar) to allow for some variation in the names, at the point when the data is being loaded. This would then allow the file to remain unchanged.

[Walter Roberson]

I find hints in the HDF5 documentation that renaming in-place might be possible -- but that might perhaps only apply to files constructed a particular way. -v7.3 MAT files are "modified" HDF5 format.

Answer 1

[OCDER]

You can use MEX code to delete a variable from a .mat file, but then you still have to save the variable via -append feature. The mex code (RemoveVariableFromMatFile) was provided in the Q&A https://www.mathworks.com/matlabcentral/answers/254048-how-can-i-delete-variables-in-my-mat-file-without-loading-them-into-matlab-7-2-r2006a

You have to compile the MEX code using a C compiler https://www.mathworks.com/matlabcentral/fileexchange/52848-matlab-support-for-mingw-w64-c-c++-compiler

mex RemoveVariableFromMatFile.c

Then you do this:

T = load('filename.mat', 'BadVarName');
GoodVarName = T.('BadVarName')
RemoveVariableFromMatFile('filename.mat', 'BadVarName');
save('filename.mat', 'GoodVarName', '-append')