Index exceeds matrix dimensions error "for my code"

I have a dataset with 1200 rows and 6 coulmns. The sixth coulmn is the dependent variable which I need to remove for the porpuse of my code. I have tried this code with many diffrent datsets and it works except this dataset and few others. I got the message "Index exceeds matrix dimensions".

        names = {'M'};
        normalisation = 1; % normalisation to unit variance
        condNumber = 10;
        nN = length(names);
        dimens = zeros(nN, 4);
        for k = 1:nN
            % Display database name
            fprintf('\n%s',names{k});
            % Load database
           % I imported my data using imported icon
            X(:,6) = []    % remove a dependent variable
            % normalise if necessary
            if normalisation == 1
                X = zscore(X);
            elseif normalisation == 2
                mi = min(X);
                ma = max(X);
                X = bsxfun(@rdivide, bsxfun(@minus, X, mi), (ma - mi));
            end
            % Calculate results
            e = svd(X) .^ 2;
            e = sort(e,'descend');
            dimens(k, 1) = sum(e >= mean(e));
            dimens(k, 2) = brokenStick(e);
            ind = find(e < (e(1) / condNumber));
            dimens(k, 3) = ind(1) - 1;
            [~, dimens(k, 4)] = SeparabilityAnalysis(X);
        end
        
end

When I mported the data, I used numeric matrix and olny loaded 1200x6 matrix and it is ok.

The peoblem with code "Index exceeds matrix dimensions", but I do not wht it works with other same datset but not with this.

Bear in mind I used others datset from the same website and large rows and columns and the code works fine.

friedman datset

https://sci2s.ugr.es/keel/dataset.php?cod=81

Can anyone guide me with this error please?

Answer 1

[Star Strider]

It is unfortunate that we do not get the opportunity to see what ‘X’ is for the various files, nor anything else about them.

If you always (and only) want to remove the last column, then:

X = X(:,1:end-1);

would work, and you can completely remove this line:

X(:,6) = []    % remove a dependent variable

that may be throwing the error.

Alternatively:

X = X(:,1:5);

might be appropriate.

We have no way of testing that to determine what may be best.

[AZ AI]

Thank you for you reply.

I tried the both solution you provide me but the problem same happens.

X is a matrix 1200x6, and I removed the sixth column.

I will provide the link of the data set.

When I mported the data, I used numeric matrix and olny loaded 1200x6 matrix and it is ok.

The peoblem with code "Index exceeds matrix dimensions", but I do not wht it works with other same datset but not with this.

Bear in mind I used others datset from the same website and large rows and columns and the code works fine.

friedman datset

https://sci2s.ugr.es/keel/dataset.php?cod=81

[Star Strider]

How are you reading the file?

I assumed that you were importing it to a double array. However, if you are importing it as a cell array, it might be a (1x1) cell, in which case you would have to do something like this:

C = {randi(9, 6)};                              % Create Cell Array
whos('C')                                       % Display Properties
D = C{:}(:, 1:5);                               % Keep Only First 5 Columns
whos('D')                                       % Display Properties

Note that the whos calls display:

  Name      Size            Bytes  Class    Attributes
  C         1x1               400  cell               
  
  Name      Size            Bytes  Class     Attributes
  D         6x5               240  double    

[AZ AI]

I read it as a Numeric Matrix

[AZ AI]

I used the same reading way for others datsets and it works

[Star Strider]

Please post the code you use to read it. I still believe that is where the problem may be.

Anyway, knowing how you read it will help me define the problem.

[AZ AI]

[AZ AI]

Sorry for the big screeshot, but it gives you an idea of how I read my datset

[Star Strider]

Do this:

opts = detectImportOptions('friedman.dat');
AAF = readtable('friedman.dat',opts);
Cols1to5 = table2array(AAF(:,1:5));

That worked for me in R2019b, and produced a (1200x5) double array in ‘Cols1to5’. Note that Column 6 has been removed. To use it:

Col6 = table2array(AAF(:,6));

That should do what you want.

[AZ AI]

I got this message "Undefined function or variable 'detectImportOptions"

[Star Strider]

The detectImportOptions function was introduced in R2016b.

This seems to work:

In = csvread('friedman.dat', 10, 0);

The ‘In’ variable is a (1200 x 6) double array.

[AZ AI]

Thanks, this works, but when I tried to remove the dependent variable which is the sixth coulmn in this datset, the code does not work.

[Star Strider]

I have no idea what you are doing and the reason you cannot get it to work.

This works for me:

In = csvread('friedman.dat', 10, 0);
Cols1to5 = In(:,1:5);
Col6 = In(:,6);
First5_Cols1to5 = Cols1to5(1:5,:)                       % View Resullts (Delete Later)
first5_Col6 = Col6(1:5,:)                               % View Resullts (Delete Later)

producing:

First5_Cols1to5 =
    0.6965    0.3584    0.4258    0.3303    0.2225
    0.5904    0.4307    0.8690    0.0709    0.6343
    0.8277    0.6178    0.9494    0.6701    0.6408
    0.8107    0.2621    0.4542    0.8547    0.2798
    0.4068    0.8162    0.8611    0.1289    0.1575

and:

first5_Col6 =
   11.0950
   13.2292
   25.3397
   15.1816
   14.4331

Use my code!

[AZ AI]

Dear Star Strider

Thanks for you support!

It is Strange! I do not know where is the exact problem.

The code is working until I removed the the sixth columns. ""Index exceeds matrix dimensions""

I have tried this with many datsets and it works for example https://sci2s.ugr.es/keel/dataset.php?cod=78

I did the same for the data in the link I removed the tenth column and it works but I do not know why is not working or the one I shared with you before.

Thank you so much

[Star Strider]

My pleasure!

I have no idea how the code you posted in your original Question figures into reading the ‘friedman.dat’ file, or what you want to do with the data in it. I am certainly willing to help you with it, however I have no idea what you want to do, beyond reading the ‘friedman.dat’ file, and we have apparently solved that problem.

I have no idea what you mean by ‘tenth column’ since the ‘friedman.dat’ file only has six columns.

If my Answer helped you solve your problem, please Accept it!

[AZ AI]

The tenth dataset is in here https://sci2s.ugr.es/keel/dataset.php?cod=78 , which works when I used the same code. The idea of what I am doing is that I calculate diffrent dimensions of the datasets. The different dimensions measured are PCA-K, means intrinsic dimension measured by PCA with Kaiser rule, PCA-BS means intrinsic dimension measured by PCA with broken stick rule, PCA-CN means intrinsic dimension measured by PCA with conditional Number rule, SepD means separability-based dimension, FracD means fractal dimension.

[Star Strider]

Are you having problems reading or working with that file?

I need to know what you want me to do with it before I download it.

[AZ AI]

I will explain briefly

The code in the original question calculates the four different dimensions I mentioned.

I used many datasets and all it works except friedman dataset and the other two. The dataset with the tenth columns works with code but friedman does not work which I do not know. In each dataset, I need to remove the dependent variable and calculate the different dimension for the independent variables (features).

I hope everthing is clear now.

[Star Strider]

O.K. So you are not having any problems reading the files.

Are you having apecific problems with the files or the data they contain?

I have not done anything with respect to fractal dimension in decades, and I did not do much with it then, so I cannot help you with that aspect of this.

[AZ AI]

The "Fractal" dimenion has another code ans is ok. The question is only for PCA-K, PCA-BS , PCA-CN , and SepD dimension.

With all datasets, I import the data and remove the dependet variable.

I can read the data file, but with some datasets the code is running (gives me number of dimesnion value for PCA-K, PCA-BS , PCA-CN, SepD) and with some others as "friedman" is not working and giving me an error message "Index exceeds matrix dimensions".

[Star Strider]

I can help you read the data files and extract the columns you want from the imported matrices.

It would appear that you may be ‘hard-coding’ something about the matrices, rather than adapting your code to the dimensions of the individual matrices. I cannot help you with the fractal dimension and related code because I have no idea what you are doing.

Answer 2

[AZ AI]

Thank you so much and I will try to see what is the problem.

[Star Strider]

As always, my pleasure!

I will help as much as I can, however I will need a clearer description of what you want to do in order to help most effectively. Just now, I have no idea.