high physicial memory usage

hello, I'm working with a script that does the folowing: 1. load a file (high reslolution audio file: 5 seconds long about 2.5mb) 2. run a apectrogram on the audio file 3. save the figure as a jpeg 4. repeat 1440 times. when i'm running the script, the physical memory that matlab is using keeps getting higher and higher untill it reaches 100% (about 35gb) and stops (crashes). my question is, why would the memory usage keep rising? and are there ways to fix this problme?

[Guillaume]

Most likely it's a bug in your code where you keep growing an array instead of reusing it. Whatever it is though, it's impossible to tell without seeing your code.

[ytzhak goussha]

thanks for your reply. one more note to add: i have split the process into smaller parts. i managed to run the script for the small portion (500 files instead of 1440 files) without reaching 100% memory usage (it reached 99%). what is interesting is that after the process was ended successfully. i cleard the workspace and Matlab stayed at high memory usage (35gb) untill i closed it.

here's the code (It's a bit embarrassing since i'm not a programmer):

fullList=dir;               %get a list of all files and folders
fullList(1:2)=[];           %detele first 2 '.','..'
counter=0;                  %total file counter
listOfFilesFull=cell(2);
listOfFolders=fullList([fullList.isdir]);   %get list of folders and thier number
lenFolders=length(listOfFolders);                       
for i=1:lenFolders                          %for each folder
    folderName=listOfFolders(i).name;       %get the name of folder
    correntFolder=dir(fullfile(folderName,'*.mat'));          %get the files in the folder
    listOfFilesInFolder=correntFolder(~[correntFolder.isdir]);%get list of files in the folder
    lenListInFolder=length(listOfFilesInFolder);
    for j=1:lenListInFolder                                    %for each file, save it's name and folder name
        listOfFilesFull{j+counter,1}=[listOfFilesInFolder(j).name];
        listOfFilesFull{j+counter,2}=[folderName];
    end
    counter=counter+lenListInFolder;         %add to file counter
end
listOfFiles=dir(fullfile('*.mat'));    %get a list of files in main folder and their number
lenListOfFiles=length(listOfFiles);         
for i=1:lenListOfFiles                      %add each file to full file list
    listOfFilesFull{counter+1,1}=listOfFiles(i).name;
    counter=counter+1;
end

for i=1:counter
    filenamecell=listOfFilesFull(i,1);
    filename=filenamecell{1};
    dirameofmatcell=listOfFilesFull(i,2);
    dirameofmat=dirameofmatcell{1};
    fullname=[dirameofmat,'/',filename];
    load (fullname)
    spectrofun2(rawClip,filename,dirameofmat)
    %call_analysisfun(rawClip,filename,dirameofmat)
    disp([dirameofmat,'/',filename])
    disp(['File ', num2str(i),' from ',num2str(counter)])
end
disp('Jobe done')

[ytzhak goussha]

:

function spectrofun2(rawClip,filename,subdirname)
%this function gets a mat clip,a file name, and a dir name
%and runs a spectrocram on it. after that it saves the figure as a jpeg
%at each sub-folder

      figure1=figure;             %open a figure
      set(figure1, 'Visible', 'off');

      spectrogram(rawClip.values,1024,512,1024,2.5E5,'yaxis');    %run the spectrogram
      %colormap jet;              %define color map
      dim = [.13 .69 0.3 .3];     %define anotaion for the figure
      str = filename;
      str2= strrep(str,'_','-'); %if there's a '_' in the name, replace it with '-'
      annotation('textbox',dim,'String',str2,'FitBoxToText','on');

      if exist('subdirname','var')
          jpegname=[str,'.jpg'];
          figname=[str,'.fig'];
          %dirname=jpegname(1:2);
          dirname='figures_no_filter';
          pathname=[dirname,'/',subdirname];
          if ~exist(pathname,'dir')        %create a folder with the hour as name
              mkdir (pathname)
          end
          fullname_jpeg=[pathname,'/',jpegname];
          fullname_fig=[pathname,'/',figname];
          saveas(figure1,fullname_jpeg)
      end

end

Answer 1

[Guillaume]

Each time you call spectrofun2 you create a new figure. You never close the figure, so if you're processing 1400 files, you'll end up with 1400 open figures. You can't see these figures since you've set their visibility to off. That's also why clear did not have any effect. close all would have had.

The fix is simple add

close(figure1)

as the last line of spectrofun2 (before the end obviously).

------

Unrelated, if you're on R2016b or later, your code can be greatly simplified as dir lets you obtain all the files in subfolders directly.

filelist = dir(*/*.mat);  %requires R2016b or later

would get all the mat files in the subfolders of the curent folder.

------

Also unrelated,

fullList(1:2)=[];           %detele first 2 '.','..'

is not a safe way of getting rid of '.' and '..' as there's no guarantee that these two will be the first two directories returned by dir (They won't be if another directory starts with any of !"#$%&'()*+,-). This is guaranteed to work:

fullList(ismember({fullList.name}, {'.', '..'})) = [];

[ytzhak goussha]

wow, thank you so much! that was very observant of you, i wouln't have figured this out alone. and thanks for the other tips. i will implement them.