Creating a multi-dimensional array out of many lower-dimensional arrays

Hello!

I currently have a hundred or so large datasets that are MxN (thus 2-D), for the external program that I use to gather this information I must run it for each of the parameters that I'm changing, for example ... etc. Therefore I now have hundreds of MxN datasets that need to be combined into a multi-dimensional array. To give an example:

What I currently have:

data(alpha,mach) = 15; % element of the data at specified alpha and mach, for delta1 = 0; delta2 = 0; delta3 = 0; delta4 = 0;...
data(alpha,mach) = 10; % element of the data at specified alpha and mach, for delta1 = 5; delta2 = 0; delta3 = 0; delta4 = 0;...

Now I would like to have

data(alpha,mach,0,0,0,0,...) = 15;
data(alpha,mach,5,0,0,0,...) = 10;

I've looked into the "cat" command, but I'm not 100% on how it works, and how to check that the data that I've combined retains the same information.

Any help would be greatly appreciated!

[Stephen23]

Based on your small example this would not be an efficient use of memory: your proposed array would have many zeros inbetween the actual values that you store. A table might be more suitable for that data:

https://www.mathworks.com/help/matlab/tables.html

Another approach is to create vectors of the delta values (so that these do not need to be contiguous) and create a numeric array for the data where the indices of the delta vectors correspond to the same indices in the data array. That would likely be a much more efficient use of memory.

[Ayden Clay]

Hi Stephen,

Pardon my ignorance, can a table still be accessed dynamically and quickly during a simulation? The total array would likely contain around a couple thousand entries, but would need to be accessed and interpolated amongst each time-step for a dynamic simulation (the data is aerodynamic).

The idea is that I can provide current information for example alpha = 5; beta = 0; mach = 2, delta1 = delta2 = delta3 = delta4 = 0; and then look into the table and interpolate for the correct drag value, say. Could this be done with tables in a more efficient manner than arrays? I've never used tables before.

I appreciate your response and look forward to your next.

Ayden

[Matt J]

How many elements will the final multi-dimensional array hold?

[Ayden Clay]

So, using the current run as an estimate (there will be other runs that may be slightly larger), it's made up of 10*12*5*5^4 = 375000 datapoints. Technically, there are also 15 of these. So in the end I would have, say:

data.aero.cd = *375,000 datapoints* ordered (alpha,mach,beta,delta1,delta2,delta3,delta4).

and equivalently for cl, ca, cn, ... etc.

[Matt J]

And in what form do you have the data (to be concatenated) now?

[Ayden Clay]

Currently the data is in many smaller datasets of the form

data.6{1} = cd @ alpha = -5:30; mach = 5:25; beta = x; delta1 = y, delta2 = z, delta3 = a, delta4 = b, ...

So the data is like 500 separate arrays running through all the permutations of the (currently) 7 parameters. Now the coefficients (say cd) are MxN arrays such that data.6{x} = (alpha,mach) = ans; where x is an index corresponding to a specific beta, delta1, delta2, ... etc. but what I'd like is cd(alpha,mach,beta,delta1,delta2,...) = ans; where those parameters are specifically part of the array.

From this I will then run my simulation and each time step calculate the alpha, mach, beta, and delta's, then use multi-dimensional interpolation to determine the coefficient of drag, from the multi-dimensional array produced here.

[Matt J]

I'm sorry, but your Maltlab syntax looks entirely made-up, making it hard to understand how your data is held. For example "data.6" is not a name that Matlab would allow you to give to any variable type. My suggestion would be that you re-write the code that created your data so that as it loops through the different delta combinations, it stores results to an N-D array directly. That's what you will want to do anyway for future runs.

[Ayden Clay]

Sadly, the program that I use is external and produces data in an unchangeable way, the data is stored in data.for006{1:end} sorry for my confusion, I misstyped.

[Taiwo Bamigboye]

Have tried so hard to learn MATLAB on my own but it seems not working for me. Please is there anyone that knows it very well and ready to teach me via zoom or team view, off course am ready to pay. All this online learning is not intuitive for me.

Answer 1

[Matt J]

If your "datasets" really are in the form of the Matlab dataset type described here, then I think you could probably just do something like the following:

C=dataset2cell(data); %convert the datasets to cell array

[m,n,p,q,r,s,t]=deal(10,12,5,5,5,5,5); %The data dimensions

dataArray = reshape( cat(3,C{:}) , [m,n,p,q,r,s,t]);

Now, for interpolation purposes, you can create a griddedInterpolant object as below,

gridVectors ={alphaList,machList,delta1List,....}; %List of sample grid values for each of the variables

lookupFunction=griddedInterpolant(gridVectors,dataArray);

[Ayden Clay]

Oh no, I'm sorry the external program that I use has an output called a "dataset" by the developers, but I don't imagine it constitutes what MATLAB has called a dataset. I have no control over how the data is entered into MATLAB, I am just given X NxM matrices, where N is the number of alphas, M is the number of machs and X is the number of betas times the number of deltas (1 through some number greater than 4). I really wish I had more control as I'd change it completely, but I'm stuck with that and would just like to combine them into a useable format for the program.

I already have the interpolation and everything set up as I knew how to do this already. But the combination of the data is something that I'm not familiar with.

[Ayden Clay]

Just because I realise the example was poor, I've generated the following code that produces similar datasets for just 1 variable, beta. How would I go about producing a 3-D array from this data, and I can work out how to extend this to the other dimensions?

alpha = -5:5:30;
mach  = 1:6:25;
beta = -5:2.5:5;

num = zeros(length(alpha),length(mach),length(beta));

for cnt = 1:numel(alpha)
    for cnt1 = 1:numel(mach)
        for cnt2 = 1:length(beta)
            num(cnt,cnt1,cnt2) = 3*alpha(cnt)+mach(cnt1)/2+beta(cnt2);
        end
    end
end
for cnt2 = 1:length(beta)
    data.for006{cnt2}.cd = num(:,:,cnt2);
end

[Matt J]

So you mean you don't have "num" and want to know how to recover it given only "data"? Then it would be as follows:

S=[data.for006{:}];

numrecov=cat(3,S.cd);

You can verify that this recovers the original num variable like so,

>> isequal(numrecov,num)

ans =

  logical

   1

In higher dimensions, you would probably need to reshape numrecov to have the dimensions that you want:

   numrecov=reshape(numrecov,[m,n,p,q,r,s,t]);

[Ayden Clay]

Thank you! This seems to be the method, I'm having some trouble with implementation, but I'm sure I'll figure that out shortly, once I have I'll mark this as the answer.

[Ayden Clay]

I have since solved this issue finally using the permute command to correctly organize the data. Sadly, this comes with the requirement that the number of betas and the number of machs are the same. I'm working on fixing this, but the working code is now:

S = [data.for006{:}];
numrecov = cat(3,S.cd);
data.aero.cd = permute(reshape(numrecov,[length(alpha),length(beta),length(mach),length(delta1),length(delta2),length(delta3),length(delta4)]),[1 3 2 4:length(RANGES{1})]);
isequal(data.aero.cd,num)

Which works for length(beta)==length(mach). I'm yet to find a solution that doesn't rely on that process.

[Matt J]

I don't see anything in your code that relies on length(beta)==length(mach).

[Ayden Clay]

It seems to be the case that the reshaped array will be size = [nalpha,nbeta,nmach,ndelta1,...]; Which, must be permuted (swapping the mach and beta dimensions) to produce the correct array. So, nmach == nbeta.

I wish it weren't the case, if the reshape method or the cat method can immeditely produce the correct array that would be perfect.