Out of memory error
5 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Yagiz Dereboy
le 9 Août 2022
Réponse apportée : Yagiz Dereboy
le 21 Août 2022
I am trying to run a optimzation problem but I am keep getting out of memory error. It's a 20k variable 300 constraint problem. I dont understand why this happens. I'll give the constraint functions and the main script respectively. This is a modified version of the p-median problem but thats not very important I guess for my question at hand.
function constraints = constraintFcnEq(x,y)
for i = 1:150
constraints(i) = sum(x(i,:)) == 1;
end
constraints(151) = sum(y) == 3;
end
function constraints = constraintFcnIneq(x,y)
for i = 1:150
constraints(i) = sum(x(:,i)) - 150*y(i) <= 0;
end
end
A = readmatrix("iris.csv");
A = A(:,2:5);
A = normalize(A);
dist_mat = pdist(A);
dist_mat = squareform(dist_mat);
x0x = zeros(150,150) + 1/150;
x0y = zeros(150,1) + 3/150;
x = optimvar("x",150,150,"LowerBound",0,"UpperBound",1);
y = optimvar("y",150,1,"LowerBound",0,"UpperBound",1);
initialPoint.x = x0x;
initialPoint.y = x0y;
problem = optimproblem;
problem.Objective = sum(dist_mat.*x,...
'all')+ 100*sum(10.*(y.^2)./(10.*(y.^2) + 0.01));
problem.Constraints.constraint1 = constraintFcnIneq(x,y);
problem.Constraints.constraint2 = constraintFcnEq(x,y);
options = optimoptions("fmincon","Display","iter","PlotFcn","optimplotfval");
show(problem);
[solution,objectiveValue,reasonSolverStopped] = solve(problem,initialPoint,...
"Solver","fmincon","Options",options);
2 commentaires
Matt J
le 9 Août 2022
You haven't provided us the means to run your code, so we can only guess. However, you might try re-expressing your constraints in a more vectorized way:
problem.Constraints.Xeq = sum(x,2)==1;
problem.Constraints.Yeq = sum(y)==3;
problem.Constraints.XYineq = sum(x,1)-150*y.'<=0;
Réponse acceptée
John D'Errico
le 9 Août 2022
Modifié(e) : John D'Errico
le 9 Août 2022
You have 20000 variables. So fmincon will be computing a Hessian matrix internally of size 20000*20000 = 4e8. But that matrix is double precision. So it will alone require 8*4e8=3.2e9. So roughly 3.2 GB of RAM will be required. And if a copy is ever made, or say, a factorization of the matrix is ever computed, then you need to immediately double the memory required. Fmincon will indeed need to factor that matrix, so you are starting to get in trouble. And that is only for that one array. Worse, MATLAB stores arrays in CONTIGUOUS blocks of memory. So you need to have enough RAM around so that MATLAB will be happy.
These days, it is still not uncommon for people to try to use MATLAB to solve problems like this on a computer with 4GB of RAM, Even 8GB of RAM would not be even close to enough here.
Plus de réponses (1)
Voir également
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!