Can't save workspace of 3 month calculation - URGENT

14 vues (au cours des 30 derniers jours)
arnold
arnold le 6 Sep 2014
Commenté : Rodrigo Bezerra le 30 Août 2019
Hi,
I did some scientific simulation on a university server which took more than 3 months. The result worked out great. I wanted to save the workspace so I could continue analyzing on a local machine.... but I can't figure it out!!
I tried
save('data.mat');
which returned
Warning: Variable 'image' cannot be saved to a MAT-file whose
version is older than 7.3.
To save this variable, use the -v7.3 switch.
Skipping...
Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ]
is not serializable
Error using save
Can't write file C:\data.mat.
then I read about v7.3 option and tried it like this:
save('data.mat','-v7.3');
which gave me
Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ]
is not serializable
Error using save
Can't write file C:\data.mat.
Now I'm at a loss. I really need this data and I can't affort to have to run the 105 days simulation again! Please help me save this workspace somehow. Estimated size is just one GB.
kind regards, Arnold

Réponse acceptée

arnold
arnold le 6 Sep 2014
Modifié(e) : arnold le 6 Sep 2014
I found a solution. There were some variables which were of class 'java.awt'. I deleted them since they don't contain anything I need.... and now it works.
Panic-attack over :)
I wonder why I have to set the parameter 'v7.3' manually in the save command, why isn't it standard?
thanks
  3 commentaires
David
David le 4 Mai 2015
where, can you provide a link? Why did you add this comment?
Rodrigo Bezerra
Rodrigo Bezerra le 30 Août 2019
Here is a suggestion to change the MATLAB configuration to save the MAT-Files using version 7.3 without setting the parameter 'v7.3'. By default MATLAB uses version 7 to save MAT-Files.
"To change your preferences to save new files in Version 7.3 format, access the Environment section on the Home tab, and click Preferences. Select MATLAB > General > MAT-Files. This preference is not available in MATLAB Online™."

Connectez-vous pour commenter.

Plus de réponses (6)

Image Analyst
Image Analyst le 6 Sep 2014
Sounds like perhaps you unwisely chose to have a variable called "image", which is the same as a built in function name. Perhaps save() is getting confused as to which image you want to save. Try renaming that variable. Also try saving things more frequently than every 3 months. At least try saving it after the first iteration so that if something goes wrong you don't have to wait 105 days to find out.
It also sounds like maybe you're trying to save new version MATLAB things into an old version .mat file, which I guess has a different format. Try deleting the existing file first and then saving it so that it creates a brand new one.

arnold
arnold le 6 Sep 2014
Thanks for your response,
of course I usually save things more frequently, but the core code of this does not provide any possibility to do so, it is provided to our reasearch group by an external partner and all the temporary files are binary. :) Anyways. I renamed "image" to "image2" and tried to save the workspace again using:
save('data.mat','-v7.3');
and again:
Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ] is
not serializable
Error using save
Can't write file C:\data.mat.
and no, there was no 'data.mat' in the root directory before. However, it always does write a file with some content. The variable 'image2' is not contained though. Checked via:
whos('-file','data.mat')
So renaming it did not solve the problem. Here's some more info on the 'image2' variable:
whos('image2')
Name Size Bytes Class Attributes
image2 1x1 2997904144 struct
>> image2
image2 =
visible: [4096x4096x3 uint8]
invalid: [4096x4096 logical]
heightmap: [4096x4096 double]
peaks: [4096x4096 logical]
confocal: [4096x4096 double]
mask: [4096x4096 double]
indices: [4096x4096 double]
pyramids: [4096x4096 double]
indices_original: [5785x5785 double]
borders: [4096x4096 double]
sizemap: [4096x4096 double]
sizemap_tmp: [4096x4096 double]
pyramid_density: [723x723 double]
pyramids_superposition: [401x401 double]
distance: [4096x4096 single]
sizemap_rgb: [4096x4096x3 double]
sizemap_overlay: [4096x4096x3 uint8]
sizemap_diameter_rgb: [4096x4096x3 double]
sizemap_diameter_overlay: [4096x4096x3 uint8]
peaks_new: [5785x5785 double]
valleys: [5785x5785 double]
fft_borders: [2705x2705 double]
Strangely enough, trying to save 'image2' on its own seems to work:
>> save('image2.mat','image2','-v7.3')
>> whos('-file','image2.mat')
Name Size Bytes Class Attributes
image2 1x1 2997904144 struct
I'd feel better having a way to properly save the entire workspace though.
Regards
Arnold
  2 commentaires
per isakson
per isakson le 6 Sep 2014
Modifié(e) : per isakson le 6 Sep 2014
"Strangely enough" &nbsp why strange? The warning indicated that it should work with the -v7.3 switch: "To save this variable, use the -v7.3 switch."
"Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ] is not serializable" &nbsp is most likely a different problem. Did you use java.awt.Robot? Could there be an instance of that Java class in the work space?
Proposal: Save a few variables at a time until you spotted the variable that causes the problem.
arnold
arnold le 6 Sep 2014
I have solved it by doing exactly that. there were several variables of class 'java.awt.Robot'. Cleared them and it worked. I'll can't fully check the code since part of it is pcode.
I find it strange that the standard behavior for the save command is not the most general one (allowing for large variables to be saved), why would it be one which is less general? So from now on to all times we should have to use the -v7.3 extension in order to save our ever growing workspaces? I find that counterintuitive and would much rather have an optional 32bit switch or something.

Connectez-vous pour commenter.


arnold
arnold le 6 Sep 2014
ehhh... why is any *.mat I save with '-v7.3' WAY bigger than without that option?
  1 commentaire
Walter Roberson
Walter Roberson le 4 Mai 2015
-v7.3 saves in HDF files, a standard format that has more overhead than the older format.
My understanding is that HDF files can sometimes have compression for components, but I do not know the circumstances under which that applies. See the discussion here and note that MATLAB might have changed which compression it does automatically since that time.

Connectez-vous pour commenter.


David
David le 4 Mai 2015
So... I'm getting this same error, trying to save a struct variable with no ROBOTS in it. I'm surprised that no one can answer the actual question here, which is why version 7.3 has anything to do with saving a brand new mat file with a brand new version of matlab.
Sadly, my 90 hour data run already closed, and all my attempts at intermediate saves errored out with this same -v7.3 garbage.
I don't want any lectures about intermediate saves, or how I should've made sure it was working before a 90 hour run. Just an answer to the question please.
  2 commentaires
per isakson
per isakson le 4 Mai 2015
Modifié(e) : per isakson le 4 Mai 2015
  • "this same error" &nbsp Please describe the error in some detail. I don't understand what you refer to.
  • "same -v7.3 garbage" &nbsp What do you mean?
  • "surprised that no one can answer the actual question" &nbsp What is the actual question?
Walter Roberson
Walter Roberson le 4 Mai 2015
You need -v7.3 to save any variable that contains an object, or to save a variable whose storage size exceeds 2 Gb.
What are the class() of the elements of your struct variable?
As to why -v7.3 is not the default: -v7.3 results in considerably larger files, and the access to data can be quite a bit slower. In early implementations, there were reports of save() requiring 2+ hours for data that took about 5 minutes to save() with -v7 . Performance improvements have been made, but the time and storage are still a fair bit more.
Most people do not need the extended facilities at all, and even then not most of the time. If a facility is only needed (say) 5% of the time, then does it make sense to penalize performance notably the other 95% of the time?
The people who do need the facility regularly, the people who routinely run with large data sets or with saving objects: those people test their code.

Connectez-vous pour commenter.


Akhilesh Thakur
Akhilesh Thakur le 8 Août 2017
Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ] is not serializable
Error using save
Can't write file
These are the errors I got today. But even using v7.3 as it says use for larger files > 2GB this didn't do any effect. I found out that there are timing issues with save. If you are calling save option again and again to update your file, there is an internal error of saving it. Even I am running a huge program that gives me result after 3 days. I found out i was using save command after every iteration of my program. This won't work. I added a functionality where I can manually save the file using flag or keypress. I realised that save command failed after calling it 19 times. I dont know what happens exactly when it is exactly called at 19th time. But the point is whenever you want to save file use a mechanism that will use this command very less frequently. So even if someone wants to save the file and the program is running, You can do a keypress So i did if( handles.keypress == 'p') save('results.mat');
So the conclusion is avoid calling save multiple times. This method worked for me.
Hope this helps.
  1 commentaire
Akhilesh Thakur
Akhilesh Thakur le 10 Août 2017
Modifié(e) : Akhilesh Thakur le 10 Août 2017
One more change
I checked only saving the data and filtering out other data such as small variables and handles, so I am only saving required variables in the result like this
save example.mat results
This works for me and never fails. Try filtering out only data which you want. Without handles and other info it is working. I guess it's handles which is messing with java.awt. This is the way where save worked for me for 192 hours of continuous calculations.
Hope this works for everyone. I spent almost 4 days debugging this.

Connectez-vous pour commenter.


Mech Princess
Mech Princess le 27 Nov 2018
So the problem is that there are some variables that are communicating with java. So once it comes to that variable, it stops saving and gives the error
not serializable
Error using save
My workspace has a lot of variables both serializable and not. Does anyone know how to save ignoring the variables not serializable without having to delete it manually or name them one by one?
Thanks

Catégories

En savoir plus sur File Operations dans Help Center et File Exchange

Produits

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by