Read a specific column of data from multiple text files and place to new variable.

Hi all,

I have a bunch of .txt data that's organized into 3 runs (meaning I have 3 text files) in a folder. As such, each text file is named differently. However, the content of each text file contains the same 9 headers, but the data for each of those 9 columns obviously differs depending on every run (meaning depending on each text file).

What I want to do is take only the data from columns 7 and 8 for EVERY TEXT FILE, and place all of the data from column 7 and column 8 into new variables called "Blue1" and "Blue2", and write this into a new text file.

The issue is when I read the data from the new columns, how do I tell at which point the data for Run1 ends and Run 2 begins? Is there a way to add variable names (row labels) to the data, aside from adding the new column variables as well?

Any help would be appreciated, still new to MatLab.

Thanks

Answer 1

[Aditya]

Here are three ways to separate data from different runs when combining specific columns from multiple text files:

1] Introduce a Delimiter After Every Run:

• You can insert a specific delimiter (like a row of NaNs or a distinct marker value) at the end of each run in the new file. This delimiter acts as a visual and logical separator between data from different runs.

% Example of adding a delimiter
delimiter = NaN; % Use NaN as a delimiter
Blue1 = [Blue1; delimiter]; % Add delimiter to Blue1
Blue2 = [Blue2; delimiter]; % Add delimiter to Blue2

2] Store the Length of Each Run in a Separate File:

• Record the number of entries for each run in a separate metadata file. This file can be referenced later to determine where each run's data starts and ends within the combined data file.

% Example of storing run lengths
runLengths = [length(column7), length(column8)]; % Store lengths of each run
save('run_lengths.txt', 'runLengths', '-ascii'); % Save to a text file

3] Use a Separate Label Column:

• Add a new column to your data that labels each row with the corresponding run number. This label column helps identify which run each data point belongs to.

% Example of adding a label column
runLabel = repmat(k, length(column7), 1); % Create a label for each row
combinedData = table(runLabel, column7, column8, 'VariableNames', {'Run', 'Blue1', 'Blue2'});

These methods provide clear ways to distinguish between data from different runs, facilitating easier data management and analysis. Choose the method that best fits your project requirements and data processing workflow.