Sparse matrices and memory

Why does this line consume several GB of my RAM

a = sparse(1,1,1,2^30,2^30, 1);

and this line results in out of memory error?

a = sparse(1,1,1,2^34,2^34, 1);

I am specifically telling matlab that I wish to store exactly one nonzero value, how can there be problem with memory? What can I do to be able to use large matrices containing very little data (<1% of RAM)? Isn't this the whole point of sparse matrices and if so, why are they so poorly implemented? Matlab 2017a. Thanks for any suggestions.

[James Tursa]

See this link for a discussion of sparse matrix storage requirements:

https://www.mathworks.com/matlabcentral/answers/126295-memory-usage-in-sparse-matrix

Answer 1

[Jan]

See: doc computer, 2nd output maxsize:

 maxsize, the maximum number of elements allowed in an array with this
 version of MATLAB.

In R2017a it is 2^48 - 1. You cannot address more elements in an array. You sparse matrix does have 1 element only, but it could be:

a = sparse(2^30, 2^30, 1, 2^30, 2^30, 1);

also, and this cannot be addressed.

The error messages are different:

a = sparse(1,1,1,2^30,2^30, 1);

 Out of memory. Type HELP MEMORY for your options.

b = sparse(1,1,1,1,2^60, 1);

 Sparse matrix sizes must be non-negative integers less than
 MAXSIZE as defined by COMPUTER.  Use HELP COMPUTER for more details.

The later message is more useful. But I assume both have the same cause.

Does this work:

a = sparse(1, 1, 1, 2^24, 2^23, 1);

This is smaller than MAXSIZE.

Answer 2

[Steven Lord]

The amount of memory required to store a sparse matrix is a function not only of the number of nonzero elements but also of the number of columns in the sparse matrix. See the documentation for the MEX API function mxSetJc, the paper "Sparse Matrices in MATLAB: Design and Implementation" by Gilbert, Moler, and Schreiber in the PDF documentation, and the description of the compressed sparse column format in the Wikipedia page on sparse matrices.

When you try to create a sparse matrix of size [2^30, 2^30] that will require a jc array of length 2^30+1 which is fairly large.