Checking file or data integrity
17 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Sasha Egan
le 19 Juil 2018
Réponse apportée : Giresse
le 23 Avr 2024 à 2:42
G'day mates, I am collecting data from a multi-channel DAQ, therefore, the datasets tend to be quite large. I am splitting each channel out from the lumped data into its own respective vector and saving an interim copy as a .mat binary. Is there a way to verify that the data saved in the .mat binary and the vector in memory are identical? i.e. ( in pseudo-code )
A = [ channel 1 ]
A_check = checksum (a)
save A as A.mat
A.mat_check = checksum(A.mat)
assert {
A_check equals A.mat_check
} throw exception e1 CheckSumsDoNotMatch
if (A_check != A.mat_check)
...try again...
Cheers.
0 commentaires
Réponse acceptée
Jan
le 20 Juil 2018
Modifié(e) : Jan
le 20 Juil 2018
A = rand(1, 1e6);
A_check = DataHash(A); % Or: GetMD5(a, 'Binary')
save('A.mat', A);
FileData = load('A.mat');
A_reloaded_check = DataHash(FileData.A);
if ~isequal(A_check, A_reloaded_check)
error('Saving failed!');
end
You should definitely stop with an error. If the saving fails, there is a severe problem, which cannot be handled reliably by repeating. Either the disk is damaged or the CPU melts down.
[EDITED] Guillaume is right: Calculating the checksums has no advantage. I suggested it only, because there might be a need anywhere, which was not mentioned in the question. An efficient version of my code would be:
A = rand(1, 1e6);
save('A.mat', A);
FileData = load('A.mat');
if ~isequal(A, FileData.A)
error('Saving failed!');
end
2 commentaires
Guillaume
le 20 Juil 2018
Modifié(e) : Guillaume
le 20 Juil 2018
I don't really see the point in computing the hashes. It's just going to take as long as simply comparing the matrices themselves. After loading the mat file, you could just do:
if ~isequal(A, FileData.A)
error(...)
What's puzzling me is why the question in the first place? If you suspect that for some reason the data does not survive the round trip through a mat file, then you should have procedures in place to detect hard disk corruption and recover from such. That should happen at the OS or hardware level, not within matlab.
matlab itself, will not corrupt a mat file.
Plus de réponses (2)
Giresse
le 23 Avr 2024 à 2:41
A = rand(1, 1e6);
A_check = DataHash(A); % Or: GetMD5(a, 'Binary')
save('A.mat', A);
FileData = load('A.mat');
A_reloaded_check = DataHash(FileData.A);
if ~isequal(A_check, A_reloaded_check)
error('Saving failed!');
end
0 commentaires
Giresse
le 23 Avr 2024 à 2:42
A = rand(1, 1e6);
A_check = DataHash(A); % Or: GetMD5(a, 'Binary')
save('A.mat', A);
FileData = load('A.mat');
A_reloaded_check = DataHash(FileData.A);
if ~isequal(A_check, A_reloaded_check)
error('Saving failed!');
end
0 commentaires
Voir également
Catégories
En savoir plus sur Get Started with MATLAB dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!