Navigating and extracting data from different folders

Hello,

I have a path of data which I would like to access various folders and various files within these folders. All the folders with the data I require to access are stored within the single folder ''Experimental data''.

Within the single folder, ''experimental data'' there is a number of participant folders labelled as follows:

Each H** folder which is the particpant number contains three seperate folders from different measurement devices. Namely research sensor, motionsense and vicon. For example if one were to open H01, three folders would be stored within it:

Within these three folders associated to each particiapnt there are a number of files which contain data of different activities carried out by the individual. For example the different tasks which are contained in the three files (for example if Vicon folder were opened) are listed as follows:

All tasks in each of the three folders vicon, motionsense and researchdevice are labelled in the same format. Each of the three folders contain the same files, with the same activities however, the data has just been collected from the specific device.

I would like to be able to compare each individual task file, T01 for instance for all H**files (the different participants) for each of the three measuremnet devices which have their data stored in the three seperate files (vicon, motionsense and research device).

In other words I would like to be able to read in the data for a specific task (specified by user) for all three measuremnet systems (vicon, motionsense and research device) for all participants (All H** folders).

I was able to access data for each participant seperately but I am unsure how to collate all the tasks to be able to compare the same task but carried out by different particpants.

Any help is hugely appreciated.

Please let me know if any clarity needs to be provided, or if existing code would be helpful?

Thanks again,

Lexi

[Johan]

Hello,

you can use the dir function to look up folder structure:

MyFolder = 'Path_to_your_folder'
MyFolderInfo = dir(strcat(MyFolder,'\**\*'))

This will give you a structure containing informations on all the files and folder inside "MyFolder".

You should then be able to discriminate your data using conditions on the folder paths and/or file name and load the data you are looking for. For example:

userinput = 'T01.xls'; %input file name

%find all file name with userinput
condition = regexp({MyFolderInfo.name},userinput); 

%create a logical mask and find the indices of valid data
index_mask = find(cellfun(@(x)~isempty(x),condition)); 

%load selected data in a working structure
data_name = regexprep(userinput,'\.\w+'); %take user input and remove all character after a .
for i_selected = 1:length(index_mask)
    %create a structure reference 
    structured_data(i_selected).(data_name) = load(fullfile(MyFolderInfo(index_mask(i_selected)).folder,MyFolderInfo(index_mask(i_selected)).name))
end

userinput = 'MotionSense'; %input file name

%find all folder name with userinput
condition = regexp({MyFolderInfo.folder},userinput); 

%repeat indexing and loading step

Using a similar logic, you should be able to select a given participant, a given task, a given measurement device and compare the data.

Hope this helps

[alexandra ligeti]

Thank you so much for this, trying to get it working in my code now. Really appreciate it.

[alexandra ligeti]

Hi again,

I am a little bit confused with what these lines are doing?

%create a logical mask and find the indices of valid data

index_mask = find(cellfun(@(x)~isempty(x),condition));

%load selected data in a working structure

data_name = regexprep(userinput,'\.\w+'); %take user input and remove all character after a .

Also sorry if this is s silly questions but see how you have * in this line :MyFolderInfo = dir(strcat(MyFolder,'\**\*'))

what is the difference between 1 * and 2 *?

Thanks,

Lexi

[Stephen23]

1 * : matches any number of characters.

2 * : matches any number of subdirectories.

See: https://www.mathworks.com/help/matlab/ref/dir.html#bup_1_c-2

Answer 1

[Image Analyst]

You can either use dir with double stars as @Johan Pelloux-Prayer mentioned in the comment above, or you can use a file data store fileDatastore

Those will get you all the file names in a convenient structure. From there you can use functions like ismember or contains to select certain files for comparison or analysis.

[alexandra ligeti]

Thank you so much!