Trying to do calculations with very large arrays

Okay this might take a lot of background and explanation, sorry. I'm trying to write a MATLAB script to automate a bunch of iteration for the design of a shaft, ive got a bunch of different variables including materials, different drives, distances between points on the shaft, bearings etc. I've been using matrices to do my calculations rather than a bunch of for loops and I'm at a point where I've got a bunch of 4D arrays. These arrays are basically used to represent the dimensions of different sections of the shaft (10 sections each with there own diameters and lengths). The issue I'm having is the final stage involves a bunch of calculations involving all of these arrays and they aren't all the same dimensions. The shaft has 2 bearings (A and B) and 2 pulleys (P and M) attached to the shaft. Some of the dimensions are basically defined as functions of the inner diameter of bearing A and are 281x81x81x81, some of them are defined as functions of the inner diameter of the pulley P and are between 12x81x81x81 and 16x81x81x81, some of them are defined as functions of the inner diameter of the bearing B and are 365x81x81x81 and some of them as functions of the inner diameter of the pulley M and are between 12x81x81x81 and 16x81x81x81 (might be different or the same dimensions as pulley P, depends on my iterative loop). The first dimension for all of these arrays is basically the number of options for each variable. E.g. there are 281 different bearings i can have at bearing A, between 12 and 16 different pulleys for Pulley etc. I need to be able to do calculations for all permutations of components. E.g. calculations using Bearing 1 at bearing A, pulley 3 at pulley P, bearing 37 and bearing B and pulley 9 at pulley B. I considered creating one enormous 7D array however, I don't exactly have several terrabytes+ of ram (i do have enough to work with what are already some decently large arrays 64gb). I'd like to know if theres a way for me to do these calculations at all? (Thanks for any help, sorry for the waffle)

[John D'Errico]

12x81x81x81? NOT remotely large. But too confusing a question to have a useful response.

My guess is what you need to do is to learn how to use tools of optimization. Here GA might be useful, since it will handle discrete problems.

[daniel]

I had to reduce them down from 12x401x401x401 because the computer started having issues with other calculations. I need the arrays to be the same dimensionally in the 2nd, 3rd and 4th dimensions. Sorry the explanation was confusing.

Answer 1

[Christopher McCausland]

Hi Daniel,

I am a little lost with the explination, however, it sounds like the the full array does not need accessed all at once. This might be a good aplication for tall(). The whole idea with tall arrays is that they can be larger than local memory and only compute as required. You can then run in sections and use gather to return the data in chunks.

I hope this helps,

Christopher

[daniel]

I think that does help, thank you very much!