Can I reduce memory requirements of a script by unloading parts of a matrix I am not using anymore in the loop?

I know this might have been asked several times already, but I couldn't find a feasible solution for my problem. I am working on intensive lattice simulations, and since I am also using explicit dynamics, I end up with a very big matrix (~20000 x 50000) to store the calculation results. This would be fine with me, but apparently my OS doesn't think so, as it kills the Matlab process randomly when he thinks the app is using too many of the system resources (no Matlab crash report, no way to retrieve data after hours of calculations!). For this reason, I started thinking about possibile ways to reduce the amount of memory the simulation requires. The solution for the "n+1" time step only requires the values of the function at time step "n", but since I use the big matrix to store all the time steps, the memory usage is enormous. Is there any way to "unload" all the previous results and only retain the last calculated state?

MWE:

time_steps = 50000; % Time steps
dof = 22000; % Degrees of freedom
H = zeros(dof,time_steps); % Matrix to Store Solution
H(:,1) = ones(22000,1); % Just numbers
for i=2:time_steps
H(:,i) = H(:,i-1)*K; % Solve for the next step (K is a known matrix)
end

[Matt J]

If K is a non-scalar matrix, then this line

H(:,i) = H(:,i-1)*K;

should be giving you an error. Possibly, you meant H(:,i) = K*H(:,i-1) ?

Answer 1

[Matt J]

Just make H a vector and keep over-writing it.

    H = ones(22000,1); % Just numbers
    for i=2:time_steps
      H = K*H; % Solve for the next step (K is a known matrix)
    end

[Alessandro Fascetti]

I need to store all the values somewhere, that's the whole purpose of the question!

[Matt J]

In each pass through the loop, you could append H incrementally to a .mat file using a matfile object. I hope you've also considered storing your matrices in sparse form if applicable.

[Walter Roberson]

Or if not a matFile then consider fwrite it to a file as you go.

[Alessandro Fascetti]

The H matrix contains all non-zero elements (sparse will not help) and I do not want to write the results on a text file because I need to load the variables for subsequent sets of analyses. The matfile object seems promising, I will look into that to see if I can get it to work.

[Matt J]

It wouldn't have to be a text file. FWRITE will let you write to a binary file as well...

[Walter Roberson]

fwrite is primarily for writing binary, and reading back from a binary file should be fast for numeric arrays.