Add commas between number columns from text file

Hello,

First time asking throught the forum.

I have a text file with some information that I want to extract as a matrix to work with later on.

The first file (ReactionsPortic5Combination4.txt) is the raw information out of another programm and the second one (5_4.txt) is the first one processed so only the rows with information remain. My problem is that the columns FZ (4th column) and MX (fifth column) are too near and when I try to extract the matrix I have a 5 column matrix with a NaN value column.

How could I add commas between the numbers for all files like this so this issue doesn't happen.

Thanks in advance.

function [Matrix] = ObtainTxtWithNumber(NameTxt,...
		NumPortic,NumComb)
    % NameTxt = "ReactionsPortic5Combination4.txt"; (in the case exposed)
    % NumPortic = 5; (in the case exposed)
    % NumComb = 4; (in the case exposed)
	NumLines = linecount(NameTxt); % This linecount function is a copied and pasted function I found 
    % in this forum. Thanks for the answers to that post and for the one
    % who asked in the post too.
    % It counts the lines that are in the text file

	
	tfile = GenerateFileName(NumPortic,NumComb);
	
	fin = fopen(NameTxt,'r');
	fout = fopen(tfile, 'w');
	% I generate another txt file and write the lines I pick
	% from the other txt
	
	
	for i = 1:NumLines
        inline = fgets(fin);
        if i > 11.5 && i < NumLines - 2.5
            fwrite(fout, inline); 
            % Write only the lines with information
        end
	end
	fclose(fin);
    fclose(fout);
    fout = fopen(tfile,'r');
	Matrix = readmatrix(tfile);% Option 1
    fclose(out);
end

[Pau Dolz Ripollés]

Sorry I forgot to say that in the last column would be better to add a semicolon.

Sorry and thanks again

Answer 1

[dpb]

fn='https://www.mathworks.com/matlabcentral/answers/uploaded_files/1367139/ReactionsPortic5Combination4.txt';
opt=detectImportOptions(fn);
fieldWidths=[8,9,18,13,13,13,13];
optFix=fixedWidthImportOptions('NumVariables',numel(opt.VariableNames), ...
    'DataLines',opt.DataLines(1), ...
    'VariableNames',opt.VariableNames, ...
    'VariableWidths',fieldWidths, ...
    'VariableTypes',opt.VariableTypes);
tData=readtable(fn,optFix)

tData = 20×7 table
    NODE      FX          FY           FZ            MX             MY          MZ  
    ____    _______    ________    __________    ___________    __________    ______

      1     -24.975     -1.3746    5.0365e-10     5.2761e-08    8.4476e-06    484.39
     19     -26.152     0.13238    3.7684e-10    -2.2521e-08    5.5713e-06    503.35
     37      -33.09    -0.69838    3.4387e-10     2.8659e-11    3.7395e-06    630.49
     55     -38.451     0.59015    3.4801e-10     9.8044e-09    3.7973e-06    729.56
     73     -45.382    0.060675    3.9078e-10      6.015e-09    4.1368e-06    853.64
     91     -54.819    -0.76315    5.0576e-10     3.2387e-09    2.9349e-06    968.62
    109     -68.239       -2.85    7.0748e-10    -1.3959e-08    6.5013e-06    1323.7
    127     -105.96     -2.9653    1.1124e-09     6.9627e-09    9.0247e-06    1862.2
    145     -140.35    -0.17742    1.1455e-09     1.6362e-08     1.219e-05    2504.2
    163     -197.26      3.1047     1.736e-09      5.696e-08    1.7765e-05    3642.9
    181     -266.73      2.7963    2.2934e-09     3.6143e-08    2.4321e-05    4873.5
    199     -352.77     -1.9058    2.8356e-09    -1.1891e-08    3.3755e-05    6575.8
    217      -477.9     -4.6443    2.8429e-09    -2.0146e-08    4.8573e-05    9164.7
    235     -684.23     -12.417    2.5996e-09    -6.5156e-08    6.5773e-05     11844
    253     -838.14     -28.256    1.2673e-09    -2.7017e-07    8.3698e-05     14327
    271     -1052.3     -4.3061    -9.081e-10    -6.1259e-08    9.7972e-05     17310

The handling of fixed-width text files in MATLAB is still fraught with far more difficulties than should be, but is a side-effect of the use of the C i/o scanning rules (instead of the much more sensible and useable FORMAT from Fortran); C simply doesn't acknowledge there is such a thing and so ignores field widths on input thinking it "knows better" than to believe what the user tells it.

The fixedWidthImportOptions object is the best thing there is so far and it's none too good; as the above shows, you either have to completely hand-code it or use the default data from the (usually reliable) import options scanning tool to locate the pieces-parts of interest to load. Trying to scan with the 'fixedwidth' option is, so far in my experience, virtually useless; it never seems to get anything right.

NOTA BENE: The above file has blanks at the RH side -- whether that's real in the original or a figment of the saving of the file for attachment -- if one uses the apparent width for the MZ column, one gets an extra empty variable on the end; you'll have to see how the real files behave.

[Pau Dolz Ripollés]

Hello, I tried to apply this solution and really worked for most of my text files but in some cases appeared an error:

Error using matlab.io.text.FixedWidthImportOptions (line 48)

Expected a vector of positive integers of the same length as

VariableNames.

Error in fixedWidthImportOptions (line 39)

opts = matlab.io.text.FixedWidthImportOptions(varargin{:});

Error in Txt2Matrix (line 5)

optFix=fixedWidthImportOptions('NumVariables',numel(opt.VariableNames),

...

fn = {
    %'ReactionsPortic5Combination1.txt';...
     %   'ReactionsPortic5Combination2.txt';...
      %  'ReactionsPortic5Combination3.txt';...
       % 'ReactionsPortic5Combination4.txt';...
        %'ReactionsPortic34Combination1.txt';...
        %'ReactionsPortic34Combination2.txt';...
        %'ReactionsPortic34Combination4.txt';...
        %'ReactionsPortic55Combination1.txt';...
        %'ReactionsPortic55Combination2.txt';...
        %'ReactionsPortic55Combination3.txt';...
        %'ReactionsPortic55Combination4.txt';...     
        'ReactionsPortic55Combination5.txt';...
        'ReactionsPortic34Combination5.txt';...
        'ReactionsPortic34Combination3.txt';...
        'ReactionsPortic5Combination5.txt'
};
for i = 1:s1
    Name1 = fn{i}
    Txt2Matrix(Name1)
end
% The ones that are omitted are the files that work. Your answer solved
% most of the cases but these four give the error previously copied.

Error in QuickMainTry (line 22)

Mat_i = Txt2Matrix(fn)

The function Txt2Matrix(fn) is more or less a copy of your solution. The first problem that happened is that some values were stored as char so I made a few arrangements so I convert them to double.

function [Matrix]=Txt2Matrix(fn)
    clear opt fieldWidths optFix tData Cell1
    opt=detectImportOptions(fn);
    fieldWidths=[8,9,18,13,13,13,13];
    optFix=fixedWidthImportOptions('NumVariables',numel(opt.VariableNames), ...
    'DataLines',opt.DataLines(1), ...
    'VariableNames',opt.VariableNames, ...
    'VariableWidths',fieldWidths, ...
    'VariableTypes',opt.VariableTypes);
    tData=readtable(fn,optFix);
    tData = tData(1:end-2,:);
    Cell1 = table2cell(tData);
    S1 = size(Cell1,1);
    S2 = size(Cell1,2);
    for i = 1:S1
        for j = 1:S2
            C1 = Cell1{i,j};
            F1 = C1;
            if ischar(F1)
                C = convertCharsToStrings(F1);
                B = str2double(C);
                Cell1{i,j} = B;
            end
        end
    end
%     Cell1
    Matrix = cell2mat(Cell1);
end

The text files which gave an error where the ones attached in this post. Thanks again!

[dpb]

The file apparently used G formatting and so the first examples didn't look like the last -- the real format appears to have been

%FORMAT(I8,1X,6G13.5)

using Fortran syntax for brevity. I wondered about that, but didn't have another file to confirm what looked unusual to have done with the first.

So instead of treating those early as blank fields, keep the correct width for all -- using @Stephen23 shorthand which is probably just as good or even better since the header length doesn't change so don't really need to try to find it nor the number of variables, use

fn='https://www.mathworks.com/matlabcentral/answers/uploaded_files/1368144/ReactionsPortic5Combination5.txt';
tData=readtable(fn,'filetype','fixedwidth', ...
    'Range',11, 'VariableWidths',[8,14,13,13,13,13,13]);
[head(tData,4);tail(tData,4)]

ans = 8×7 table
    NODE        FX            FY           FZ             MX            MY            MZ    
    ____    ___________    ________    ___________    __________    ___________    _________

      1     -0.00071439    0.019427     1.4392e-14    8.5328e-12      3.585e-10     0.065325
     19     -0.00069382    0.033387     6.6242e-15     3.711e-11     2.3173e-10     0.070759
     37     -0.00059587    0.049676    -8.2662e-16    4.2049e-11     1.0698e-10     0.037428
     55     -0.00049219     0.04852    -2.6255e-15    4.0731e-11     6.5034e-11     0.035304
    289      0.00074026     0.05027     4.3874e-15    8.2018e-11    -9.0868e-11    -0.031066
    307      0.00075688    0.037125      1.233e-14    4.8502e-11    -1.1089e-10    -0.075261
    NaN             NaN         NaN            NaN           NaN            NaN          NaN
    NaN       0.0011316     0.89264       1.23e-21    1.1196e-09      2.655e-10     0.011049

and all should be well for all files. The the fixed-width object doesn't have an 'X' format to skip a column so the first uses the width of 14 instead of 13 as there is a blank column before it.

You'll want to set the error handling properties to skip the bad records so it doesn't pick up the lines after the given line, too, of course.

[dpb]

fn = {
    %'ReactionsPortic5Combination1.txt';...
     %   'ReactionsPortic5Combination2.txt';...
      %  'ReactionsPortic5Combination3.txt';...
       % 'ReactionsPortic5Combination4.txt';...
        %'ReactionsPortic34Combination1.txt';...
        %'ReactionsPortic34Combination2.txt';...
        %'ReactionsPortic34Combination4.txt';...
        %'ReactionsPortic55Combination1.txt';...
        %'ReactionsPortic55Combination2.txt';...
        %'ReactionsPortic55Combination3.txt';...
        %'ReactionsPortic55Combination4.txt';...     
        'ReactionsPortic55Combination5.txt';...
        'ReactionsPortic34Combination5.txt';...
        'ReactionsPortic34Combination3.txt';...
        'ReactionsPortic5Combination5.txt'
};
for i = 1:s1
    Name1 = fn{i}
    Txt2Matrix(Name1)
end

Do something more like...

dataDir='YourDataDirectoryLocation';
d=fullfile(dataDir,'ReactionsPortic*.txt');         % return dir struct of wanted files
for i=1:numel(d)
  tData(i)=readDataFunc(fullfile(d(i).folder,d(i)name));    % process each file in turn
end

[dpb]

function [Matrix]=Txt2Matrix(fn)
    clear opt fieldWidths optFix tData Cell1
    

Remove clear from function -- a function has its own context space, none of those variables exist in the function until they're created which haven't yet done. It's pointless and waste of time; doesn't do anything.

Once you have the table, use it -- don't go through all those gyrations converting it to something else.

Depending upon just what the next step(s) are, the table may well have builtin facilities to do the analyses wanted/need automatically. We don't know what those are, but somebody undoubtedly could/would point out ways to do whatever it is if were shown that.

IFF, however ("the big if" and only if) it really is needed to have a matrix, then use readmatrix with the later-shown fixed-width import object and don't use a table. But, it's often very convenient to have things in tables instead; just depends on what is to be done.

Either way, however, it takes only the one or two lines to get whichever is the desired route; not any such gyrations as shown above.

Answer 2

[dpb]

Given the fixed format nature of these files, the best thing to do is probably to just build a specific fixed-width import options object for them that is hand-built to match the specific format.

The following should work for all your files regardless the actual data format in the fields; using the %g format for such may be somewhat convenient to the human reader, but makes parsing a little more problematic unless and until one sees the actual format in enough incarnations to recognize what it is.

fn='https://www.mathworks.com/matlabcentral/answers/uploaded_files/1368144/ReactionsPortic5Combination5.txt';
opt=fixedWidthImportOptions('NumVariables',7, ...
                            'VariableWidths',[8,14,13,13,13,13,13], ...
                            'VariableTypes',repmat({'double'},1,7), ...
                            'VariableNamesLine',11, ...
                            'DataLines',12, ...
                            'PartialFieldRule','omitrow', ...
                            'MissingRule','omitrow');
tData=readtable(fn,opt);
[head(tData,4);tail(tData,4)]

ans = 8×7 table
    NODE        FX            FY           FZ             MX            MY           Var7   
    ____    ___________    ________    ___________    __________    ___________    _________

      1     -0.00071439    0.019427     1.4392e-14    8.5328e-12      3.585e-10     0.065325
     19     -0.00069382    0.033387     6.6242e-15     3.711e-11     2.3173e-10     0.070759
     37     -0.00059587    0.049676    -8.2662e-16    4.2049e-11     1.0698e-10     0.037428
     55     -0.00049219     0.04852    -2.6255e-15    4.0731e-11     6.5034e-11     0.035304
    253      0.00069967    0.046202     8.1348e-16    7.5452e-11    -8.7023e-11     -0.03168
    271      0.00072792    0.046072      3.015e-15    5.7354e-11    -8.1638e-11    -0.022533
    289      0.00074026     0.05027     4.3874e-15    8.2018e-11    -9.0868e-11    -0.031066
    307      0.00075688    0.037125      1.233e-14    4.8502e-11    -1.1089e-10    -0.075261