How to speed up this code?

Hi all,

I have several CANBUS .ASC files that I have to convert to .MAT files for further analysis. In particolar, I need a struct which contains some specified signals where the list is contained in a variable called Signals2Load, which in turn they are contained in a list of messages named Message2Load. Due to the fact that the functions canSignalImport and canSignalTimetable do not work (I am in contact the MathWorks support for this), I had to create my own code. Below, you will find the code. Unfortunately, it is pretty slow for a 1h log (more than 3e6 messages to analyse), it takes almost two hours for the conversion. What could I do for speeding up the code?

Thanks.

Best regards,

Michele

    message=canMessageImport([ASCFiles(iASC).folder '\' ASCFiles(iASC).name],'Vector',candb,'OutputFormat','timetable');
    messageStruct=table2struct(message);
    Duration=floor(seconds(message.Time(end)));
    CanData(iASC).Time=[0:DT:Duration];
    Message2Sel=find(contains({messageStruct.Name},Message2Load));
    if length(unique({messageStruct(Message2Sel).Name}))~=length(Message2Load)
        warning(['The following messages were not found ']);
        setdiff(Message2Load,unique({messageStruct(Message2Sel).Name}))
    end

    for i=1:length(Message2Sel)
        Sig=fieldnames(messageStruct(Message2Sel(i)).Signals);
        [Sig2Sel iSig2Load iSig]=intersect(Signals2Load,Sig);
        if isempty(iSig2Load)==0                           
            for iSigFound=1:length(Sig2Sel)
                Sigs{iSig2Load(iSigFound)}=[Sigs{iSig2Load(iSigFound)}; messageStruct(Message2Sel(i)).Signals.(Sig{iSig(iSigFound)})];
                Time{iSig2Load(iSigFound)}=[Time{iSig2Load(iSigFound)}; seconds(message.Time(Message2Sel(i)))];
            end
        end
    end

[lvn]

Did you profile the code? (type "profile on" before running the code, then let it run for a couple of minutes, then type "profile report"). This will allow you to see which lines take up most room (and might help others to provide help).

[Serbring]

Yes, the real problem is the number of interations inside the for. In a case, more than 2 millions of iterations has to be carried out.

[lvn]

Yes, but which for, and which line is most expensive?

[Serbring]

Here you can see the result of the profiler.

EDIT: external link removed and image uploaded.

A line where I can do a little improvement is replancing the line with intersect using ismember, but, I have found that ismember together with find is not faster than intersect.

[lvn]

Could you also show the output of the profiler when you click on "FromASCtoMATv3"? (so that we can see line per line the time?)

[Serbring]

Here we go.

EDIT: external link removed and image uploaded.

I do not understand, in which line the Tabular.dotParenReference is called in my code. Considering the number of iterations, I suppose that this is caused by this line:

             Sigs{iSig2Load(iSigFound)}=[Sigs{iSig2Load(iSigFound)}; single(messageStruct(Message2Sel(i)).Signals.(Sig{iSig(iSigFound)}))];

[Serbring]

Hi all,

I have updated the function and the code speed has improved, but it is still slow. In the attachment, you will see the results of the code profile. As you can see the slow processing time is caused by "matlab self time (built-in, overhead,...)".

According to the profile results, this line is the critical code line:

Pos2Paste=min(find(isnan(Sigs{iSig2Load(iSigFound)})))

which I have improved a bit by substituing it with this line (i do not have the results of the profile because the code is still running):

Pos2Paste=find(isnan(Sigs{iSig2Load(iSigFound)}),1); %(*)

As I understand, this line is critical because there are many calls and many function calls. I need this function, because, the signals in the messageStruct have a different timestep and therefore, I need a cell array rather then a simple array. So what I do is saving the signal found in each message in the first NaN element of each Sigs cell ellement.

Any suggestion for a further increase of the code is welcome.

Below an updated version of my code (the line is line (*) is not in this code and niether is the result of the profile.

for iASC=1:length(ASCFiles)    
    message=canMessageImport([ASCFiles(iASC).folder '\' ASCFiles(iASC).name],'Vector',candb,'OutputFormat','timetable');
    messageStruct=table2struct(message);
    MessTime=message.Time;
    Duration=floor(seconds(MessTime(end)));
    clear message
    CanData(iASC).Time=(0:DT:Duration);
    Message2Sel=find(contains({messageStruct.Name},Messages2Load));

    UniqueMessFound=unique({messageStruct(Message2Sel).Name});
    UniqueMessageCount=countcats(categorical({messageStruct(Message2Sel).Name},UniqueMessFound));
    for iSig=1:size(Signals2Load,1)
        Time{iSig}=NaN(UniqueMessageCount(strcmp(UniqueMessFound,Signals2Load{iSig,2})),1);
        Sigs{iSig}=NaN(size(Time{iSig}));
    end
        
    
    for i=1:length(Message2Sel)
        Sig=fieldnames(messageStruct(Message2Sel(i)).Signals);
        [Sig2Sel , iSig2Load , iSig]=intersect(Signals2Load,Sig);
        if isempty(iSig2Load)==0                           
            for iSigFound=1:length(Sig2Sel)
                Pos2Paste=min(find(isnan(Sigs{iSig2Load(iSigFound)})));
                try
                    Sigs{iSig2Load(iSigFound)}(Pos2Paste)=single(messageStruct(Message2Sel(i)).Signals.(Sig{iSig(iSigFound)}));
                catch
                    i;
                end
                Time{iSig2Load(iSigFound)}(Pos2Paste)=single(seconds(MessTime(Message2Sel(i))));
            end
        end
    end
end