Fastest Way to Keep the last N Outputs of a Loop in a Matrix

3 vues (au cours des 30 derniers jours)
Hakan Caldag
Hakan Caldag le 28 Fév 2023
Commenté : Jan le 1 Mar 2023
Hey everyone, I have some code with a for loop where I would like to store the last N outcomes and do calculations intermittently with the last N evaluations within the loop. I am storing each evaluation result in a column of the matrix mydata. When the number of steps passes N, I remove the first column of data (as it is the oldest) and add my result at N+1th step to the final column as following:
mydata(:,1)=[];
mydata(:,N)=newdata;
While this works fine, it is very time-consuming. Is there a faster way to remove the first column of data and add the new data to the last column of my matrix? Note that I cannot keep all the data as it would end up being a very large matrix.
  4 commentaires
Hakan Caldag
Hakan Caldag le 28 Fév 2023
@dpb I tried your solution but unfortunately there is not much improvement.
@Jan It is a for loop. The for loop lasts, let's say 2e4 iterations and I would like to have the last 3000 results every time.
The circular buffer is a great idea, will try implementing that.
dpb
dpb le 28 Fév 2023
Not surprising but thought just outside chance the optimizer might be able to make it not too bad...not much lost to try and I had to go at the time...

Connectez-vous pour commenter.

Réponse acceptée

Jan
Jan le 28 Fév 2023
Modifié(e) : Jan le 28 Fév 2023
If you do not know the number of iterations in advance, a circulare buffer is more efficient than shrinking an expanding an array repeatedly:
N = 10;
Buffer = nan(N, 5);
iBuffer = 0;
while rand > 0.05
data = rand(1, 5);
iBuffer = mod(iBuffer, N) + 1; % Wrap around cursor
Buffer(iBuffer, :) = data;
end
LastData = Buffer([iBuffer+1:N, 1:iBuffer], :);
  4 commentaires
Hakan Caldag
Hakan Caldag le 28 Fév 2023
What I wanted was to have the latest N datapoints at each step (whenever k is greater than N). This code seems to work only for when k is larger than offset. In any case, using the circular buffer worked for me already, I just wanted to note this down for anyone that may come across this page in the future.
Jan
Jan le 1 Mar 2023
The circular buffer is filled in each iteration, but only the last N iterations matter. If the number of iterations is known in advance, it is cheaper to fill the buffer only in the last N iterations. The offset is adjusted dynamically to match the number of iterations and the wanted length of the buffer.
The circular buffer is working also, but it wastes time for overwriting memory.

Connectez-vous pour commenter.

Plus de réponses (0)

Catégories

En savoir plus sur Loops and Conditional Statements dans Help Center et File Exchange

Produits


Version

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by