Can rectangularized data be achieved by modifying a text file with MATLAB?

I have a space delimited text file with the following contents:

NFL58  23Mar2012  Show  2  1  01  0000000001  Low

001  25.187466  156.162447  21578.188  97.134234  stops  AAAAA  1  A10A

21600.123  21612.122

3100.435  4100.380  5100.739

-0.491736  1.491492  0.891808

0.051748  -0.071254  0.021175

1.100000e-01  1.200000e-01  1.300000e-01  1.400000e-01  1.500000e-01  1.600000e-01  1.700000e-01  1.800000e-01  1.900000e-01
2.100000e-01  2.200000e-01  2.300000e-01  2.400000e-01  2.500000e-01  2.600000e-01  2.700000e-01  2.800000e-01  2.900000e-01
3.100000e-01  3.200000e-01  3.300000e-01  3.400000e-01  3.500000e-01  3.600000e-01  3.700000e-01  3.800000e-01  3.900000e-01
4.100000e-01  4.200000e-01  4.300000e-01  4.400000e-01  4.500000e-01  4.600000e-01  4.700000e-01  4.800000e-01  4.900000e-01
5.100000e-01  5.200000e-01  5.300000e-01  5.400000e-01  5.500000e-01  5.600000e-01  5.700000e-01  5.800000e-01  5.900000e-01
6.100000e-01  6.200000e-01  6.300000e-01  6.400000e-01  6.500000e-01  6.600000e-01  6.700000e-01  6.800000e-01  6.900000e-01
7.100000e-01  7.200000e-01  7.300000e-01  7.400000e-01  7.500000e-01  7.600000e-01  7.700000e-01  7.800000e-01  7.900000e-01
8.100000e-01  8.200000e-01  8.300000e-01  8.400000e-01  8.500000e-01  8.600000e-01  8.700000e-01  8.800000e-01  8.900000e-01
9.100000e-01  9.200000e-01  9.300000e-01  9.400000e-01  9.500000e-01  9.600000e-01  9.700000e-01  9.800000e-01  9.900000e-01

002  25.287466  156.162447  21578.288  97.234234  stop  BBBBB  2  A10B

21600.223  21612.222

3200.435  4200.380  5200.739

-0.492736  1.492492  0.892180

0.052748  -0.072254  0.022175

1.120000e-01  1.200000e-01  1.300000e-01  1.400000e-01  1.500000e-01  1.600000e-01  1.700000e-01  1.800000e-01  1.900000e-01
2.120000e-01  2.200000e-01  2.300000e-01  2.400000e-01  2.500000e-01  2.600000e-01  2.700000e-01  2.800000e-01  2.900000e-01
3.120000e-01  3.200000e-01  3.300000e-01  3.400000e-01  3.500000e-01  3.600000e-01  3.700000e-01  3.800000e-01  3.900000e-01
4.120000e-01  4.200000e-01  4.300000e-01  4.400000e-01  4.500000e-01  4.600000e-01  4.700000e-01  4.800000e-01  4.900000e-01
5.120000e-01  5.200000e-01  5.300000e-01  5.400000e-01  5.500000e-01  5.600000e-01  5.700000e-01  5.800000e-01  5.900000e-01
6.120000e-01  6.200000e-01  6.300000e-01  6.400000e-01  6.500000e-01  6.600000e-01  6.700000e-01  6.800000e-01  6.900000e-01
7.120000e-01  7.200000e-01  7.300000e-01  7.400000e-01  7.500000e-01  7.600000e-01  7.700000e-01  7.800000e-01  7.900000e-01
8.120000e-01  8.200000e-01  8.300000e-01  8.400000e-01  8.500000e-01  8.600000e-01  8.700000e-01  8.800000e-01  8.900000e-01
9.120000e-01  9.200000e-01  9.300000e-01  9.400000e-01  9.500000e-01  9.600000e-01  9.700000e-01  9.800000e-01  9.900000e-01

Where line 1 is a header, and the remaining lines represent repeatable data blocks. I'm looking to open the file in MATLAB, rectangularize each data block as a 14x9 matrix for specific plots, and save the file.

Is this possible with MATLAB?

Thank you.

[Walter Roberson]

The obvious data block appears to be 9 x 9. Where would the other (5 x 9) = 45 values come from?

[Ahmed A. Selman]

Explain the second line, please:

001 25.187466 156.162447 21578.188 97.134234 stops AAAAA ..etc

is it all needed as a text or you intend to use the info there in a specific way? Anyway try the

fscanf

function, it can read pre-formatted data from a file, exactly as you desire i.e. numeric, text, precision, ..etc.

[Brad]

Walter, Ahmed, my apologies for the lack of clarity. These non-rectangular text files are more complicated than I thought.

Each data block has 14 lines, beginning with a number (001, 002, etc). The tool used to create these data blocks saves the data to a space delimited text file in the format you see above.

I’m successfully using TEXTSCAN to read in the 9x9 matrices into cell arrays, and I need to include the data in the 4 proceeding lines with each 9x9 matrix (resulting in a 13x9 matrix). Unfortunately, the number of elements in these 4 rows is not equal to the number of elements in the 9x9 matrix. And when I step through the following commands, I get the following error messages:

??? Error using ==> cat CAT arguments dimensions are not consistent. Error in ==> cell2mat at 81 m{n} = cat(2,c{n,:}); Error in ==> TestScript3 at 61 Data{Block,1}=cell2mat(InputText);

My MATLAB code is as follows:

% Open text file

fid = fopen('BlockDataNoZeros.txt','r');

% Read Header Row (Row 1) as a string delimited by a carriage return

InputText=textscan(fid,'%s',1,'delimiter','\n');

Headerline=InputText{1};

disp(Headerline);

% Initialize The Block Counter

Block = 1;

% Initialize the s variable s = '';

% Lets read in each data block using TEXTSCAN for each line of text % in the file

while ~feof(fid)

    % Initialize the Number Of Columns Counter
    NumCols=InputText{1};

    % Read Sequence Number Row (Row 2) delimited by a carriage return
    InputText=textscan(fid,'%f %f %f %f %f %5.0s %5.0s %f %5.0s',1,'delimiter','\n');  
    Sequence_Number=InputText{1};
    disp(Sequence_Number); 

    % Create format string for 9x9 covariance matrix
    FormatString=repmat('%f %f %f %f %f %f %f %f %f',1);

    % Read 9 X 9 matrix data (and the 4 lines prior to it)
    InputText=textscan(fid,FormatString,13,'delimiter','\n');  

    % Convert to numerical array from cell
    Data{Block,1}=cell2mat(InputText);  

    %Size Of Table
    [NumRows,NumCols]=size(Data{Block}); 

    % Increment the Block Counter
    Block = Block+1;

    % Ensure we're at the end of the file and not stuck in a while loop
    if isempty(line),break, end
    s = strvcat(s,line);

end

% Dispaly contents of s to the screen

disp(s)

%Close Text File fclose(fid);

Note that if I manually enter zeroes to the 4 lines proceeding the 9x9 matrix,such that each line has 9 elements, the code works. With several hundred data blocks, manually entering zeroes isn't practical. Is there a way to open the data file, add the applicable zeroes, and save the file (in MATLAB)?

Answer 1

[Ahmed A. Selman]

Hi Brad. Instead of

   InputText=textscan(fid,'%s',1,'delimiter','\n');

Try this out

   [InputText, dn1]=fscanf(fid,'%c', [1 50]);

if the length of your file is as written in the original question, your header (1-BY-50) should look like this

   InputText =

   NFL58  23Mar2012  Show  2  1  01  0000000001  Low

I honestly didn't get what is the size of your header. Anyhow, "Try simplest solutions first" .. and it is quite simpler if you wrote a code to read info from a file as: -text as a matrix of char, and -numbers as matrix of numbers only (2-D array).

Also try fscanf function (returns output as a matrix) instead of textscan (returns a cell array). Matrices are so much easier to handle than cell arrays.

However, if you tried something as:

   [data,nd2]=fscanf(fid,'%g', [1 inf]);

which starts reading numeric data from the file, from where it stopped the last time, you'll get this:

   data=
   1.0e+004 *

       0.0001    0.0025    0.0156    2.1578    0.0097

meaning the code stopped at encountering a char ( it is 'stops') because it is not numeric (floating-point number type, declared with %g). So you need to isolate the header from the rest, or at least isolate char from numbers in the first few lines.

I think you got the point to finish the rest.

Regards.