programmatically determine the "variablenamesline" value for import options.

My situation:

I have a piece of equipment from which the user can export data in csv format. The top rows of the file contain values for various settings. Below that section are columns of data at the top of which is a header row. The default header values for each column are always the same. I want to import the data in the columns.

My problem:

Depending on the instrument settings when the files were exported there are a different number of rows of settings values. This means that the location of the data header row can be different from file to file. I have attached images of two different files. In one the data header row is row 40 (Capture.JPF) and the other it is row 41 (Capture2.JPG).

My question:

How can I find the row that contains the correct header value (the one with my variable names) and then set the variablenamesline property accordingly so that I always import my data regardless of how the user had the unit configured at the time of exporting the data? Can I use detectImportOptions() and readTable() to do so? That is what I have been trying. Is there a better way?

Note: I am certain that this question has already been answered somewhere. I'm sorry if I'm repeating it. However I have already put in about two and a half hours searching mathworks, the internet in general and the documentation for an answer and so now am turning to you all directly. Thanks again for your help.

Details:

My Matlab version is

MATLAB Version: 9.10.0.1710957 (R2021a) Update 4

MATLAB License Number: XXXXXXXX

Operating System: Microsoft Windows 10 Pro Version 10.0 (Build 19045)

Java Version: Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

Thanks in advance for your help.

[Stephen23]

@Calvin Harrington: please upload those CSV files by clicking the paperclip button.

[Walter Roberson]

The example files both appear to use the same variable names. Are the variable names in the files all the same or do they differ? Because if they are all the same, there are other ways to approach this.

[Calvin Harrington]

The device default variable names are always "Frequency [Hz]" and "Magnitude [dBm]".

The data I want are the Frequency and the Magnitude data vectors. I don't use the settings in the upper rows. Essentially I'd like to just find the cell with the word "Frequency" in it and call that the start cell for parameters I'll use to define the number of columns and rows to import. Does that clarify? I could also define my, matlab syntax compliant, variable names during import.

Thanks for your help.

Answer 1

[Walter Roberson]

readmatrix() of the file. Use rmmissing() to remove the header lines (which would have mostly translatted into NaN)

[Calvin Harrington]

Thanks Walter. I'll give that a try, using readmatrix() with that rmmissing() setting, in the morning and let you all know.

Thanks again!

[Calvin Harrington]

Hi Walter. Thanks again. That worked. I also found an issue on my end. My default, start level, datalines value was set too high. When I just set it to [1 inf] the,

opts.ImportErrorRule = "omitrow";

setting accomplished the same thing. I guess it's possible I could someday encounter a data set with only numeric values in a row outside of the dataset but this really helps me out.

Cheers!

Answer 2

[Oguz Kaan Hancioglu]

In both figures, the sturcture of each csv files are the same. Frequency and magnitude are in the same row. I suggest to read all csv files into a table. After that, you can find the index of the where the "frequency" text in the table. You can do cell or string comparision to find the index.

[Calvin Harrington]

Thanks Oguz,

I should clairfy, the variables "Frequency [Hz]" and "Magnitude [dBm]" are indeed on the same line as each other. My problem is that from file to file that line is at a different location. In one snapshot it is line 40 and in the other line 41.

Thanks for noticing that. One doesn't always know what's clear until another reads it. :)