Saving cell to file to save RAM

I am trying to run this code to give me my values in a cell, however i am dealing with a large amount of data and it usually crashes around the 6th iteration of j. I was wondering is there any way i can optimise this or save to file line by line to save RAM. I am very new to matlab so any help is appreciated.

j = 1;              
while j <= 4              
%inputdir = 'C:\Users\jorda\Desktop\Physics Project\Country PM25';
%outputfile = 'Africa_With_Dust';   %do not put it into inputdir !
%dinfo = dir( fullfile(inputdir, '*.mat') );
%filenames = fullfile( inputdir, {dinfo.name} );
PM_val = ncread(With_Dust_List(j),'PM25');                   
i = 1;
j = j + 1;
    while i <= 51
        mat_zer=zeros(2600,7200);
        isin1 = inpolygon(LON,LAT,S(i).X,S(i).Y);
        mat_zer(isin1)=st(i);
        [row1,col1] = find(mat_zer~=0);
        mat_zer_pm = PM_val;
        inds1 = sub2ind(size(mat_zer_pm),row1,col1);
        new_mat1 = nan(size(mat_zer_pm));
        new_mat1(inds1) = mat_zer_pm(inds1);
        Country_With{i,j} = new_mat1;
        i = i + 1;
    end
%for K = 1 : length(filenames)
    %thisfile = load(filenames{K});
    %save(outfile, '-struct', 'Country_With{1:51,j-1}', '-append');
%end
end

Answer 1

[Deepak]

Hi @Jordan Coombs,

To my understanding, you have written a MATLAB code to process a large amount of input data and store the output data in a “.mat” file. However, the code crashes on the 6th iteration of the for loop as the output data is being appended to a cell array in each iteration.

To optimize the code, we can append the output data to a mat file in each iteration, that will keep the size of the output cell array limited. Additionally, we can clear the unnecessary variables after each iteration to save the memory.

Below is the updated MATLAB code that addresses the task:

inputdir = 'C:\Users\jorda\Desktop\Physics Project\Country PM25';
outputfile = 'Africa_With_Dust.mat'; % Ensure this is not in inputdir!

j = 1;

while j <= 4
    PM_val = ncread(With_Dust_List(j), 'PM25');
    j = j + 1;

    for i = 1:51
        mat_zer = zeros(2600, 7200);

        isin1 = inpolygon(LON, LAT, S(i).X, S(i).Y);
        mat_zer(isin1) = st(i);

        [row1, col1] = find(mat_zer ~= 0);

        % Initialize the PM value matrix and new matrix
        mat_zer_pm = PM_val;
        inds1 = sub2ind(size(mat_zer_pm), row1, col1);
        new_mat1 = nan(size(mat_zer_pm));
        new_mat1(inds1) = mat_zer_pm(inds1);

        % Save the data for the current region and dataset
        varName = ['Country_With_', num2str(i), '_', num2str(j-1)];
        save(outputfile, varName, 'new_mat1', '-append');

        % Clear variables to free memory
        clear mat_zer mat_zer_pm new_mat1 row1 col1 inds1;
    end
end

Please find attached the documentations of “save” and “clear” function in MATLAB for reference:

www.mathworks.com/help/matlab/ref/save.html

www.mathworks.com/help/matlab/ref/clear.html

I hope this resolves the issue.

[Stephen23]

What does varName refer to?