Count Peaks, csv-files

Hey there! I'm a clueless beginner when it comes to Matlab. So if you would be so kind and answer my questions as simple as possible. I have many problems with what I want to do in Matlab:

1. Item one Import csv-files in a loop. My idea:

    for k=1:NDatei 
    if ~exist(sprintf('20140512-0013_%03d.csv',k), 'file') 
        errordlg(['File ' num2str(k) ' does not exit'],'File Error'); 
        break ; 
    end 
    data=importdata(sprintf('20140512-0013_%03d.csv',k)) ; 
    end

Whereas my data has names like '20140512-0013_001.csv', '20140512-0013_002.csv',.. Problem is that Matlab tells me that my files don't exist. I think it's a problem with the path. Do you know why Matlab doesn't find the files although they are in a subfolder of the Matlab folder?

1. Item two Findpeaks: I want to count the Peaks in each file with a certain threshold and minimal peak distance. Apparently Matlab has some problems with my imported files, because I get the Error

Error using findpeaks
Expected X to be one of these types:

double, single, uint8, uint16, uint32, uint64, int8, int16, int32, int64

Instead its type was char.

Error in findpeaks>parse_inputs (line 90)
validateattributes(Xin,{'numeric'},{'nonempty','real','vector'},...

Error in findpeaks (line 71)
[X,Ph,Pd,Th,Np,Str,infIdx] = parse_inputs(Xin,varargin{:});

Thank you so much for your help. I will attach the files as an example. I'm really looking forward to your answers and thoughts. Cheerio! Tanja

Answer 1

[dpb]

Roger mentioned addpath to solve the subdirectory problem. To read a series of files, see the FAQ

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

I recommend the dir solution almost always. In short for your case...

 d=dir('2014*.csv');  % get the list of .csv files beginning w/ 2014
 for i=1:length(d)    % loop over the list
   [dat,txt]=xlsread(d(i).name);   % read the ith file
   % do whatever with the data here...
   ...
end

For a single file I used one of your links to get here...

>> d=dir('2014*.csv');  % get the list of .csv files beginning w/ 2014
>> [dat,txt]=xlsread(d.name);
>> whos dat
Name          Size             Bytes  Class     Attributes
dat       25004x2             400064  double              
>> whos txt
Name      Size            Bytes  Class    Attributes
txt       2x2               276  cell               
>> txt
txt = 
  'Zeit'    'Kanal A'
  '(us)'    '(V)'    
>> dat(1:3,:)
ans =
 -3.9008    0.1575
 -3.8968    0.1969
 -3.8928    0.1575
>>

Your problems on data types will go away when you read the data as above and use the numeric data array dat (or whatever you choose as a variable name instead of dat, of course).

And, if you know the headings and don't care, just return the numeric data...

dat=xlsread(d(i).name);   % read, return numeric only

[Tanja]

First of all: Thank you so much for you answer, I can tell you made an effort giving me an understandable answer. I tried to use your advice. The Problem is that Matlab doesn't give me any feedback. The list d is empty. And would you mind explaining me dat=xlsread(d.name)? What's the use of 'name' in that context? and where is the loop over k? My code so far:

addpath('/path_of_folder_containing_files');
d=dir('20140512*.csv');
for k=1:length(d)
    if ~exist(sprintf('20140512-0013_%03d.csv',k), 'file') 
        errordlg(['File ' num2str(k) ' does not exit'],'File Error'); 
        break ; 
    end
    dat=xlsread(d(k).name);
    [pks,len]=findpeaks(dat)

end

You can tell that I used your additional comment and cut off the headings, they are always the same, so I don't need them.

Thank you so much for your help in advance. Tanja

[dpb]

Use the addpath only once in the command window outside your script.

In all likelihood the form you have as

addpath('/path_of_folder_containing_files');

isn't right with the '/' in the path name. You need to look at the result after executing addpath with matlabpath to be certain it's correct for your directory structure. Once you've gotten that straightened out, use dir at the command line to ensure that you are finding the files.

If after

d=dir('20140512*.csv');

d is empty, then the choices are --

1) the matlabpath isn't right yet or

2) the wildcard doesn't match any files in a subdirectory on the path

Until, obviously, you get that resolved, nothing else is going to happen. :)

for k=1:length(d)
  if ~exist(sprintf('20140512-0013_%03d.csv',k), 'file')

Once you switch to the dir, there's no need for any such test logic -- if it doesn't exist, it won't be in the list to start with. Get rid of that whole if...end clause entirely.

    dat=xlsread(d(k).name);

dir returns a structure, the filename is the .name field within that structure. If you don't understand that, then read the section on them in the documentation at

doc struct

In the sample I did here, I only saved one file so I knew a priori there was going to be only one entry returned by dir and so I didn't need a loop nor the subscript to reference the file name. In your case, you're intending to process more than one so you expect a structure array to be returned.

[Tanja]

Thanks again for you help. Unfortunately I still need a bit of help. After trying to fix the add path problem, I figured out that all the relevant folders are listed in Matlab 'Set Path'. Is it okay to assume that the first possible problem isn't the one causing mine? After trying dir in the command line, it's still empty. How can I fix the second problem with the wildcard? Is there a way to test if that's the problem? Sorry I have no idea how to work with Matlab and to be honest I'm quite confused. Thanks again for your help! Tanja

[dpb]

OK, yes...I've slightly misled you, sorry.

dir doesn't use matlabpath; only looks in the cwd (current working directory). What addpath does is enable a subsequent file opening command like xlsread or others to open the file based only on the file name, not a fully qualified name (a name containing the path, not just relative path or the filename alone).

So, your first part needs to be

d=dir(fullfile('/path_of_folder_containing_files','20140512*.csv'));

THEN you should find d isn't empty. Same way with dir at the command line to confirm.

Sorry I forgot to mention the above; I only dabble w/ Matlab to respond here any more being "retired" from active consulting and back on the family farm as my "real day job" so I just keep all the stuff in a local \work directory rather than having to deal with real files and organization of actual projects any longer. Consequently I occasionally overlook a detail like this...

[Tanja]

No worries! I'm just soooo happy that you try to help me. Good news is that Matlab is now responding with a 43x1 struct for d. Bad news is that the xlsread has a problem with the csv files.

• Item one "Error using xlsread (line 247)Unable to read XLS file/Users/.../20140512-0013_001.csv.File is not in recognized format.Error in A20140526_Test_addpath_xlsread (line 3) dat=xlsread(d(k).name);"

[dpb]

That's progress! :) That implies it isn't a standard csv file. You're not in an area that uses commas for decimal points, by any chance? Matlab can't handle that; you'll need to convert to the US convention, if so, sorry.

Probably simplest would be to link one of your data files so can see if that's permissible from your end...

[Tanja]

I attached one of my files. I think that it uses points for decimal und commas to separate columns, for example '-3.90116676,0.15748030'. I mean Matlab doesn't have a problem if I import the files manually. Again-many thanks.

[Tanja]

I tried it with importdata instead of xlsread and it works just fine. Now I have another question if you don't mind giving me some more of your attention and help: Hab can I address a certain column in a 1x1 struct with 3 fields where one field contains 2500x2 double. From the field with 2500 doubles I want to address one column in the find peaks function. OR -if it's possible- both columns

d=dir(fullfile('/Users/.../20140512-0013_Testdaten','20140512*.csv'));
for k=1:length(d)
    dat=importdata(d(k).name);
    [pks,len]=findpeaks(dat);
end

doesn't work. pks should address one column and len the other. Any idea? Thanks again!

[dpb]

I don't understand that with xlsread -- which release of Matlab on what platform?

I just did it again here and R2012b on Windows reproduced the previous example.

Anyway, to your question--if you're using importdata, I'd just follow the step to read it in w/ one to convert the data to a double array outside the structure. Since you didn't care about the text anyway, just overwrite the structure--

dat=importdata(d(k).name);
dat=dat.data;                % turn dat into a double array

Unfortunately, the findpeaks routine works only on a vector, not an array so you'll have to operate on each column in data individually since the output of each case won't necessarily be the same, it would take a cell array to hold the answers. TMW chose not to make that extension.

You'll have to write sotoo--

for i=1:2
  [pks,locs]=findpeaks(dat(:,i));
  % do something with this one here before next pass...
  ...
end

You could create a cell array and store the results therein if need both simultaneously.

[Tanja]

I'm using R2014a for Mac. Thank you so so so much! So far everything works just fine. For my needs I can work with separate columns. Just one last question: Do you know how I can combine the threshold and the minpeakdistance in find peaks?

[dpb]

...using R2014a for Mac

Being as that is a newer release, I'd check for any updates patches that your installation is owed and if the symptom for xlsread is still there (or there are no notes that there's any patches for it) I suggest it's a bug and should send the file and bug report to TMW at mathworks.com

ADDENDUM

I whiffed on the 'Mac' part previous go-'round -- from the doc for xlsread

...
'basic' Flag to request reading in BASIC mode, which is the default for
        systems without Excel for Windows.  In BASIC mode, xlsread:
        *Only reads XLS or XLSX files.

So, it's documented behavior that .csv files aren't supported on non-Windows platforms. Too bad the error messages aren't OS-sensitive to include that piece of trivia...

Answer 2

[dpb]

...how I can combine the threshold and the minpeakdistance in find peaks?

They're named-value pairs, along w/ the rest of the options--just string them out as needed...

[pks,locs]=findpeaks(data,'minpeakdistance',3, 'threshold', pi, 'npeaks', 42);

etc., ...

Answer 3

[Roger Wohlwend]

You probably cannot open the files because the folder is not on the MATLAB search path. In your code add the following line at the beginning of the skript or function:

addpath(Path_of_the_folder_that_contains_the_csvFiles)

If you do that, Matlab will find the files and open them.

Concerning the csv files: Matlab has a problem with the first two lines that contain text. That is why it is not possible to use the function csvread. You can, however, use the function xlsread.

[~,~,raw] = xlsread('20140512-0013_001.csv')

Now delete the first three rows because they contain no numerical data:

raw = raw(4:end)

Extracting the data is, however, not so simple:

T = length(raw);
Data = NaN(T,2);
for k = 1 : T
   C = strsplit(raw{k},',');
   Data(k,1) = str2double(C{1});
   Data(k,2) = str2double(C{2});
end

Now you have a matrix Data with the numerical data of the file. Use that matrix for any computations.

[dpb]

You can, however, use the function xlsread.

[~,~,raw] = xlsread('20140512-0013_001.csv')

That's the hard way to use xlsread for the purpose, however..use the form

[dat,txt] = xlsread(filename);

instead. Then the numeric data will be in the array dat and the header text in txt automagically.

Or, use importdata that can transparently find the header lines.

Or, use textread with the 'headerlines' option--

dat=textread(filename,[],'headerlines','3);