Large Text File Import

I have 4GB text file included 9 headerlines, 24 columns of raw number data (cca 9M rows). How can I import data to matlab, separate columns and save them as MAT easily and quickly? I tried to use textscan but it gave me only first 266 rows.

fid = fopen('2013067b.txt') Out = textscan(fid,'%f %s %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f','delimiter', '\t', 'HeaderLines',9); fclose(fid)

Thank You All!

Answer 1

[dpb]

 c=reshape(textread('2013067b.txt'),'%f',-1,'delimiter','\t','headerlines',9),24,[])';

Sometimes the older tools are the better...

Answer 2

[per isakson]

"[...]textscan but it gave me only first 266 rows" Without seeing the error message it's difficult to say why that happened. There was a message? I guess there is an anomaly in line 267

.

The second column is text, %s, - or ? I missed that. However, you write "24 columns of raw number data". The examples below assumes numerical data.

Possibly, dlmread is an alternative. Try

    Mi = reshape( magic(12), [], 3 );
    fid = fopen('cssm.txt','w');
    for jj = 1 : 9
        fprintf( fid, 'Header line %i\n', jj );
    end
    fprintf( fid, '%f,%f,%f\n', transpose(Mi) );
    fclose( fid );
    Mo = dlmread('cssm.txt',',',9,0);
    all(Mi==Mo)

Or is it too slow?

.

Three more alternatives:

    fid = fopen('cssm.txt','r');
    for jj = 1 : 9
        str = fgetl( fid );
    end
    num = fscanf( fid, '%f,%f,%f\n', [3,inf] );
    fclose( fid );
    Mf = transpose( num );
    num = textread( 'cssm.txt','%f', -1, 'delimiter',',', 'headerlines',9 );
    Mtr = transpose( reshape( num, 3, [] ) );
    fid = fopen('cssm.txt','r');
    num = textscan( fid,'%f', inf, 'delimiter',',', 'headerlines',9 );
    Mts = transpose( reshape( num{:}, 3, [] ) );
    fclose( fid );
    all(Mf==Mi)
    all(Mtr==Mi)
    all(Mts==Mi)