How to save (serialize) MATLAB variables to a byte string?

Ideally this would be using the same format as the save command (.mat file), but instead of writing to disk directly, it would write it to data in memory.

The reason is I need to access the saved data directly to implement a custom hashing method on the variable, and have no need to write it to disk in order to achieve this, and would rather not go through the administratively-heavy option of creating a ram disk through ramfs.

I'd actually be OK with a version-change of MATLAB invalidating the object in my particular case (the idea being that any software updates may influence results, however unlikely).

Assuming full .mat support is not an option, I still need fairly general support, which is why the (intermediate) output of save seems like a good option: e.g., nested structs, cells, arrays. I could probably do without Java objects in this particular case.

[xd]

I wish Matlab support a method for

Serializing/deserializing Matlab data to/from memory

[Walter Roberson]

@xd

There are some objects that cannot be serialized, including:

• TCP and UDP sockets
• COM port or .NET state
• open files
• java objects
• memory pointers to C or C++ objects

[Yair Altman]

@Walter Roberson using getArrayFromByteStream/getByteStreamFromArray users can indeed serialize some elements in your list, except some internal ephemeral properties which get default values upon deserialization.

For example:

>> jb = javax.swing.JButton('Click here')
jb =
javax.swing.JButton[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.5,border=com.sun.java.swing.plaf.windows.XPStyle$XPEmptyBorder@5a4ed68f,flags=296,maximumSize=,minimumSize=,preferredSize=,defaultIcon=,disabledIcon=,disabledSelectedIcon=,margin=javax.swing.plaf.InsetsUIResource[top=2,left=14,bottom=2,right=14],paintBorder=true,paintFocus=true,pressedIcon=,rolloverEnabled=true,rolloverIcon=,rolloverSelectedIcon=,selectedIcon=,text=Click here,defaultCapable=true]

>> objectByteArray = getByteStreamFromArray(jb);  % 1x4600 uint8 array

>> jb2 = getArrayFromByteStream(objectByteArray)
jb2 =
javax.swing.JButton[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.5,border=com.sun.java.swing.plaf.windows.XPStyle$XPEmptyBorder@5a4ed68f,flags=296,maximumSize=,minimumSize=,preferredSize=,defaultIcon=,disabledIcon=,disabledSelectedIcon=,margin=javax.swing.plaf.InsetsUIResource[top=2,left=14,bottom=2,right=14],paintBorder=true,paintFocus=true,pressedIcon=,rolloverEnabled=true,rolloverIcon=,rolloverSelectedIcon=,selectedIcon=,text=Click here,defaultCapable=true]

>> char(jb2.getText)
ans =
    'Click here'

[Walter Roberson]

@Yair Altman

fid = fopen('test.txt', 'w');
fprintf(fid, 'hello\n');
ftell(fid)

ans = 6

objectByteArray = getByteStreamFromArray(fid);
fclose(fid)

ans = 0

fid = getArrayFromByteStream(objectByteArray)

fid = 3

ftell(fid)

Error using ftell
Invalid file identifier. Use fopen to generate a valid file identifier.

[Yair Altman]

Dear @Walter Roberson

The file ID returned by fopen() is a numeric integer that is automatically allocated by the operating system. It is a sort of a pointer index to the file in the OS's list of open files (or -1 upon error) - in your example the value 3 (values 0-2 are pre-selected for stdout,stdin and stderr respectively). This pointer data is ephemeral, so although it can be serialized (just as any other integer value), it has no meaning upon deserialization. Importantly, the file ID is not an object that stores internal properties that could be serialized/deserialized, and it doesn't include the file contents or meta-data. On the other hand, if you use object representations of files (e.g. System.IO.File or java.io.File), you should be able to serialize them excluding their internal ephemeral properties.

Stated in other words, even in your example, serialization and deserialization of the file ID worked as expected, in the sense that it serialized and restored the integer value "3". The interpretation of this value as belonging to an open file is not something that is inherently stored in the integer value, and so it cannot be restored. As with any programming function or engineering tool, it is up to the user to ensure that they understand what the function/tool is doing and what its limitations are. Specifically, serializing ephemeral data, while technically possible, results in garbage. Also, deserializing data requires the user to reinterpret any data which is not inherently deserialized.

Answer 1

[Mohammad Sami]

There is an undocumented function called "getByteStreamFromArray" which converts a matlab object to bytestream.

To convert it back use the opposite function "getArrayFromByteStream".

[Brandon Barker]

Very nice. If this is indeed the underlying function used by save (and maybe even if it isn't), I'd like to request Mathworks consider making it a documented function. This sort of thing can be quite useful, and even if the exact output isn't stable, as long as the type signature is stable I hope that would be enough.

[Yair Altman]

Here's a detailed discussion/explanation of the undocumented interface: http://undocumentedmatlab.com/articles/serializing-deserializing-matlab-data