Fasten the speed of assigning values to a large tensor
4 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Hi! Now I have a for-loop assigning values to some entries of a large tensor at every iteration. But it is too slow since the tensor is large. Now I am thinking if there is any way to fasten this speed. One simple example is shown below. Here I used 'ndSparse' function to create a four dimensional tensor instead of 'zeros' function. The link of 'ndSparse' is https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays. ndSparse generalizes 'sparse' function from sparse matrix to high dimensional sparse tensor.
% Simple example
M = 500;
A = ndSparse.build([M,M,M,M]);
n = 10;
for i=1:n
disp(i)
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
Is there any way to make it faster?
Thank you very much!
5 commentaires
Réponses (2)
Jan
le 28 Avr 2021
Modifié(e) : Jan
le 28 Avr 2021
Avoid creating index vectors explicitly:
idx1 = a1:b1;
A(idx1, :) = tmp; % Slow
A(a1:b1, :) = tmp; % Faster
In the first case, each element of idx1 is tested, if it is a valid index:
% For each index something like this is performed:
if idx(1) == flor(idx(1)) && idx(1) > 0 && idx(1) < size(A, 1)
With a1:b1 Matlab checks the validity of the first and last element only.
Please post a minimal working example. Before the readers can test, if their suggestion is faster, they currently have to implement your code based on guesses, what
A = create a sparse all zero 6-D tensor of size (M,M,M,M,M,M);
or
idx1 = a1:b1; (b1-a1=m)
explicitly means.
2 commentaires
Jan
le 3 Mai 2021
@Jason Yang: You are right, ndSparse is infact an exception, because this user defined class is not supoorted by Matlab's JIT acceleration.
Matt J
le 3 Mai 2021
Modifié(e) : Matt J
le 3 Mai 2021
You can save some time if you pre-allocate the array:
% Simple example
M = 400;
n = 10;
tic
A = ndSparse.spalloc([M,M,M,M],(M/n)^4*n); %Pre-allocate
%A=ndSparse.build([M,M,M,M]);
for i=1:n
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
toc%Elapsed time is 1.282638 seconds.
I don't think there are great gains to be made, however. Most of the computation is being spent just generating the entries for the array:
clear B
tic;
[B{1:4}]=ndgrid(1:M/n);
B = reshape( cat(5,B{:}) ,[],4);
[C,vals]=deal(cell(n,1));
for i=1:n
C{i}=B+M/n*(i-1);
vals{i} = randn(prod([M/n,M/n,M/n,M/n]),1);
end
C=cell2mat(C); vals=cell2mat(vals);
I=sub2ind([M,M,M,M],C(:,1),C(:,2),C(:,3));
J=C(:,4);
toc; %Elapsed time is 1.499094 seconds.
tic
S=sparse(I,J,vals, M^3,M, (M/n)^4*n);
toc; %Elapsed time is 0.344042 seconds.
tic;
A=ndSparse(S,[M,M,M,M]);
toc;%Elapsed time is 0.016531 seconds.
0 commentaires
Voir également
Catégories
En savoir plus sur Data Type Conversion dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!