How can I filter multiple files ?

I have made a filter as following

[b,a] = butter(9,.02,'low'); x=textread('noisy.out'); y=filter(b,a,x); save filtered.out y -ASCII

I have 1000 files and I would like to filter all my files using the above filter. Is there any way that I can do this efficiently? my files are named like 1.out 2.out , ... 1000.out

any suggestion would be appreciated.

my *.out files just include one column with numbers

this is one example -4.034E+02 -4.007E+02 -3.987E+02 -3.978E+02

All numbers in one column and each file contains 13000 numbers

Answer 1

[Oleg Komarov]

Parse the folder for existing 1-1000.out files and import them one-by-one apply filter and save:

% Retrieve all the files in a directory
names = dir('C:\Users\Oleg\Desktop\Nuova cartella');
names = {names(~[names.isdir]).name};

% Select only the files #.out
idx   = ~cellfun('isempty',regexp(names,'\d+\.out'));
names = names(idx);

% Filter
[b,a] = butter(9,.02,'low'); 
for n = names
  % Open file for reading
  fid = fopen([myDir n{:}],'r');
  x   = textscan(fid,'%f');
  y = filter(b,a,cat(1,x{:}));
  % Discard content and write filtered data
  fid = fopen([myDir n{:}],'w');
  fprintf(fid,'%E\r\n',y);
  fid = fclose(fid);
end

Oleg

[maadi]

I do not want to import them one-by-one. I wanna get this done by single run. I got 1000 files and I have to repeat this for many times.

[Walter Roberson]

There is no Mathworks provided way to import multiple files in a single mathworks provided routine. You can write your own routine that does a loop importing them one by one, but the effect will be little different than Oleg's loop.

If you _do_ read all of the files at one time, and if they are all exactly the same number of data points, then you can store the data as columns in an array, and then you can filter(b,a,TheArray) to do each of the columns. Or store the data as rows and specify the dim of 2 in the filter() call (columns is more efficient though.)

Your question was not clear as to the file names you want to store the filtered results in to, and not clear whether you wanted to store one output file per input file or just one large output file containing all of the filtered data.

[maadi]

Let me make it clear

each signal is put in a text file and I have named them 1.out 2.out . . . 1000.out

I want to filter my signals and removes noise and after that I name my files again 1.out 2.out .... 1000.out

I just wanna get rid of noise in my signals.

I can not put all my files in single file because every file contains 13000 data and if I put them all in one file it becomes very huge and my computer will not be able to process that

Thanks

[Walter Roberson]

The loop that Oleg shows will read in each of the files in turn; you do not need to repeat it or manually run it for each file.

[Oleg Komarov]

@maadi: it's standard procedure the one outlined above as Walter confirms. Each loop just imports one file, filters it and save the filtered version to a file (may also overwrite it). This operation is repeated for each in your folder that begins with a number and has .out extension.

There's no other way to do it with an essentially different method.

1.Import data

2.Work on data

3.Export data

Repeat 1,2,3 till no more data is available.

[maadi]

Oleg,

I put all my *.out files in a folder.

When I run your program I get this error

??? Function 'exist' is not defined for values of class 'cell'.

Error in ==> exist at 41

[varargout{1:nargout}] = builtin('exist', varargin{:});

Error in ==> textread at 160

if (exist(varargin{1}) ~= 2 | exist(fullfile(cd,varargin{1})) ~= 2) & ~isempty(which(varargin{1}))

Error in ==> Oleg at 12

x = textread(n);

why is that?

[Oleg Komarov]

You have to specify the format:

textread(filename,format)

I edited the solution above. Format should be a string.

[maadi]

I am very sorry I am new at MATLAB.

I run the following m file

clc

clear all

% Retrieve all the files in a directory

names = dir('d:\MATLAB7\work');

names = {names(~[names.isdir]).name};

% Select only the files #.out

idx = ~cellfun('isempty',regexp(names,'\d+\.out'));

names = names(idx);

% Filter

[b,a] = butter(9,.02,'low');

for n = names

x = textread(n,format);

y = filter(b,a,x);

% save...

end

But now I get this error

??? Error using ==> format

Too many output arguments.

Error in ==> Oleg at 12

x = textread(n,format);

My feeling is something is wrong with filename n

can you also change your program to overwrite every file after filtration?

Thanks

[Oleg Komarov]

So, to get it right you have to specify the format as (for example): '%9c %6s %*f %2d %3s'.

I can help here, but I need to see, say, the first three lines of a sample #.out file (headers included). Maybe edit your main post adding these lines so that I can specify the format and add the saving part.

[Walter Roberson]

And to clarify, the immediate reason why it said "Too many output arguments" is that "format" is a Matlab command that does not return any value.

(This is the sort of reason I use variables with uppercase when I mean the user to fill in values.)

[Oleg Komarov]

Ok I modified the script accordingly. Be careful: the content is imported, filtered and overwritten. Make some test files.

[maadi]

Thank you Oleg I really appreciate that