How can I grab only certain files from a folder?

Hello.

I currenlty have a folder labeled "2021-06-14". Inside are .csv files from the entire day in 15 minute increments labeled "Data 2021-06-14 00-00.csv, Data 2021-06-14 00-15.csv, ... , Data 2021-06-14 23-45.csv". My goal is to, programmatically, pull files 1-x and copy them to another folder.

Any ideas?

[dpb]

..." programmatically, pull files 1-x"

OK, too cryptic for me. What is the specific set of files wanted?

Just write an appropriate wildcard pattern and use dir()

[Brian Gregory]

Hi, sorry about that.

Basically I want the first 8 hours of data .. so "Data 2021-06-14 00-00.csv to Data 2021-06-14 07-45.csv".

I want to easily be able to copy these files to a new folder (already created) without having to change the date each day.

Essentially something like this is what I tried but it didn't work.

date = "2021-06-14";

csv1 = csvread("Data "+date+" 00-00.csv");
.
.
.
csvn = csvread("Data "+date+" 07-45.csv");

Where n is the total number of csv files. This did not work though but if I can either get this to work or write a code to pull the first n files from a folder and copy it to another folder that will work for me.

Answer 1

[LO]

first create a list of the files, then check which one has the second last digit pair lower than 08, then copy it to a given folder

% this allow you to move to the desired folder and pick a file
[logfile, pathname] = uigetfile('*.csv','Pick a csv file'); 
cd(pathname) 
% the only reason for this would be to select the folder, but you may as well comment 
% these 2 first lines and just position yourself in the folder right away and then execute 
% the lines below
pathname=pwd; % this is just to make sure the current path is copied
new_folder = ('c:\where_I_want_my_file_to_go');
file_list = dir ('*.csv') ; %create a list of csv files based on folder content
conversion = transpose(struct2cell(file_list));
name_list = (conversion(:,1));

for i = 1:height(name_list)
    filename = (name_list{i}); % this is the current filename to consider
    timing = filename(end-8:end-4); % get the last 5 elements of your file name
    timing(timing < '0' | timing > '9') = []; % this will remove special chars from the string
    value = sscanf(timing, '%d');
    if value(1:2) < 08 % consider only the first 2 digits
        copyfile(fullfile(pathname,filename),new_folder,'f')
    end
end

alternative solution:

path1 ='C:\path_to_csv_files'
path2 = 'C:\new_path'
source =fullfile(path1,'*.csv');
myfile= dir(source);

for i = 1:numel(myfile)  
    filename = myfile(i).name;
    timing = filename(end-8:end-4); % get the last 5 elements of your file name
    timing(timing < '0' | timing > '9') = []; % this will remove special chars from the string
    value = sscanf(timing, '%d');
    if value(1:2) < 08 % consider only the first 2 digits
        copyfile(myfile(i).name,path2)
    end    
end

[Brian Gregory]

Thank you for your help. The first solution does not work, honestly I do not understand it so I can't explain why it won't work. I just run it and it pulls up the file application and expects me to pick the file manually, which is not what I wish to do.

The second solution returns an error that says:

Index exceeds the number of array elements (1)

Any ideas?

[Brian Gregory]

So it turns out I only needed the copyfile line and the filename line of that for loop. So it now works. Thanks!

[LO]

if you do not want to pick files manually then remove the first two lines

% this allow you to move to the desired folder and pick a file
[logfile, pathname] = uigetfile('*.csv','Pick a csv file'); 
cd(pathname) 

regarding the second at which line the errors pops up ?

change this line

if value(1:2) < 08 % consider only the first 2 digits

to

    if ~isempty(value) & value(1:2) < 08 % consider only the first 2 digits

this script assumes that your filenames are always ending with a sequence of 5 digits (or even 4), the first 2 of which are the hour, the second 2 are the minutes.

The variable "value" contains the time digits (if it is 7.45 the value should be 0745, just type value in the command line to double check). Basically the line that I pasted here above applies the condition "value should be less than 8. this would copy all files basically recorded until 7.45.

If you want to include 8 just use "value(1:2) <= 08" instead of "value(1:2) < 08"

[Brian Gregory]

They pop up during the second if else statement

[LO]

yes the problem is the value variable is empty if your filenames do not contain the time numbers at the same place. Or otherwise I do not know what the issue could be.