i have provided one zip file with raw file in the name of"CODG0240.11I"and an output format file in the excel format.how can i extract the data from the raw file in the form of given output format.please provide the code.

please provide the code

[Guillaume]

Your raw file is a binary file. Unless you have an exact description of how the data is stored in that binary file, it's unlikely that somebody will be able to help you.

While it may possible to reverse engineer the format from your excel file, it's at least several hours worth of work, if not days.

[TAPAS]

I have to extract data for 24 hours as given in the output xls format, each excel for one hr data.read the raw file,the data starts after the end of headerlines.

[Guillaume]

Again, your raw file is a binary file. Unless you know how the data is encoded into that file it is very difficult to near impossible to work out the encoding.

[TAPAS]

I don't know how the file is encoded.is it possible to know how it is encoded looking it's data,if so then u can open the file in notepad++ and see the data

[Walter Roberson]

The zip includes CODG0240.11I.Z which is itself a compressed file. It uncompresses to a plain text file that starts with

       1.0            IONOSPHERE MAPS     GNSS                IONEX VERSION / TYPE
  ADDNEQ2 V5.1        AIUB                28-JAN-11 20:11     PGM / RUN BY / DATE 
  CODE'S GLOBAL IONOSPHERE MAPS FOR DAY 024, 2011             COMMENT             
  Global ionosphere maps (GIM) are generated on a daily basis DESCRIPTION         
  at CODE using data from about 150 GPS sites of the IGS and  DESCRIPTION         
  other institutions. The vertical total electron content     DESCRIPTION         
  (VTEC) is modeled in a solar-geomagnetic reference frame    DESCRIPTION         
  using a spherical harmonics expansion up to degree and      DESCRIPTION

After about 500-ish lines of headers it starts to have sections like

2011     1    24     0     0     0                        EPOCH OF CURRENT MAP
  87.5-180.0 180.0   5.0 450.0                            LAT/LON1/LON2/DLON/H
 13   13   13   13   13   13   13   13   13   13   13   13   13   12   12   12
 11   11   11   10   10    9    9    9    8    8    7    7    6    6    6    5
  5    4    4    4    3    3    3    3    3    3    2    2    2    2    3    3
  3    3    3    4    4    5    5    5    6    6    7    7    8    9    9   10
 10   10   11   11   12   12   12   13   13
  85.0-180.0 180.0   5.0 450.0                            LAT/LON1/LON2/DLON/H
 18   18   18   18   18   18   18   18   18   17   17   17   16   16   15   15
 14   13   13   12   11   10    9    9    8    7    6    6    5    4    4    3
  2    2    1    1    0    0    0    0    0    0    0    0    0    0    0    0
  0    0    0    0    1    2    2    3    4    5    7    8    9   10   11   12
 13   14   14   15   16   16   17   17   18

and it is those values that are recorded in the .xls files. The first column of the .xls corresponds to the "87.5" or "85.0" or so on, and the second column goes from -180.0 to +180.0 in steps of 5.0, and the third column are the 13's and 9's and so on.

What is marked as .xls files are really just text files with this data.

Doing the extraction should not be all that difficult.

The boundary to look to start recording for any one .xls file is

START OF TEC MAP

and run to the following END OF TEC MAP

[TAPAS]

How can I find how it is encoded.

[Walter Roberson]

The *.Z file is not encoded: it is compressed with the Unix "compress" utility. Most unzip programs can handle it without difficulty.

[TAPAS]

so please help me out how to extract data from it

[Walter Roberson]

Are you using Mac, or Linux? If so you can use "uncompress" to open the .Z file. If you are using Windows see https://fileinfo.com/extension/z and https://www.ncbi.nlm.nih.gov/Ftp/uncompress.html

I will wait for you to confirm you are able to get the text version of the .Z file before describing the text processing code.

[TAPAS]

this is the text format

Answer 1

[Walter Roberson]

   filename = 'CODG0240.txt';
   S = fileread(filename);

parts = regexp(S, '.*END OF TEC MAP.*', 'split', 'dotexceptnewline');
if isempty(parts{end}); parts(end) = []; end
parts = cellfun(@(str) regexprep(str, '^.*EPOCH OF CURRENT MAP[^\n]*', ''), parts, 'uniform', 0);

Now parts is a cell array of character vectors. Each character vectors is one epoch. The character vector contains repeated blocks of the form

    87.5-180.0 180.0   5.0 450.0                            LAT/LON1/LON2/DLON/H
   13   13   13   13   13   13   13   13   13   13   13   13   13   12   12   12
   11   11   11   10   10    9    9    9    8    8    7    7    6    6    6    5
    5    4    4    4    3    3    3    3    3    3    2    2    2    2    3    3
    3    3    3    4    4    5    5    5    6    6    7    7    8    9    9   10
   10   10   11   11   12   12   12   13   13

with no header before that, and no trailing line after that.

Now,

fmt = ['%.1f%.1f%.1f%.1f%.1f%*s', repmat('%f',1,73)];

result = cellfun(@(P) cell2mat(textscan(P, fmt, 'collect', true, 'whitespace', ' \n')), parts, 'uniform', 0);

Result will now be a cell array, each entry of which corresponds to one epoch in the file. Each cell array entry will be a numeric array which will have one row for each block that looks like the above -- that is, one row for each latitude longitude combination. The row will start with 5 entries corresponding to the 87.5-180.0 180.0 5.0 450.0 type of information. The string is not stored as it is the same in all blocks. The next 73 entries in the row are the 13, 13, ... etc through to the 12 13 13 -- those 5 numeric lines are bunched together. So that is 5 entries of header information and 73 numbers, total 78 per row.

[TAPAS]

but there should be data for 24 hour but its showing only for 14 hour