gpuArray

Array stored on GPU

Description

A gpuArray object represents an array stored in GPU memory. A large number of functions in MATLAB^® and in other toolboxes support gpuArray objects, allowing you to run your code on GPUs with minimal changes to the code. To work with gpuArray objects, use any gpuArray-enabled MATLAB function such as fft, mtimes or mldivide. To find a full list of gpuArray-enabled functions in MATLAB and in other toolboxes, see GPU-supported functions. For more information, see Run MATLAB Functions on a GPU.

If you want to retrieve the array from the GPU, for example when using a function that does not support gpuArray objects, use the gather function.

Note

You can load MAT files containing gpuArray data as in-memory arrays when a GPU is not available. A gpuArray object loaded without a GPU is limited and you cannot use it for computations. To use a gpuArray object loaded without a GPU, retrieve the contents using gather.

Creation

Use gpuArray to convert an array in the MATLAB workspace into a gpuArray object. Some MATLAB functions also allow you to create gpuArray objects directly. For more information, see Establish Arrays on a GPU.

Syntax

G = gpuArray(X)

Description

G = gpuArray(X) copies the array X to the GPU and returns a gpuArray object.

Input Arguments

X — Array

numeric array | logical array

Array to transfer to the GPU, specified as a numeric or logical array. The GPU device must have sufficient free memory to store the data. If X is already a gpuArray object, gpuArray outputs X unchanged.

You can also transfer sparse numeric arrays to the GPU.

Example: G = gpuArray(magic(3));

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | logical
Complex Number Support: Yes

Object Functions

arrayfun	Apply function to each element of array on GPU
gather	Transfer distributed array, Composite object, or gpuArray object to local workspace
pagefun	Apply function to each page of distributed or GPU array

There are several methods for examining the characteristics of a gpuArray object. Most behave like the MATLAB functions of the same name.

isgpuarray	Determine whether input is gpuArray
existsOnGPU	Determine if gpuArray or CUDAKernel is available on GPU
isUnderlyingType	Determine whether input has specified underlying data type
ndims	Number of array dimensions
size	Array size
underlyingType	Type of underlying data determining array behavior

Several MATLAB toolboxes include functions with gpuArray support. To view lists of all functions in these toolboxes that support gpuArray objects, use the links in the following table. Functions in the lists with information indicators have limitations or usage notes specific to running the function on a GPU. You can check the usage notes and limitations in the Extended Capabilities section of the function reference page. For information about updates to individual gpuArray-enabled functions, see the release notes.

Toolbox Name	List of Functions with gpuArray Support	GPU-Specific Documentation
MATLAB	Functions with gpuArray support
Statistics and Machine Learning Toolbox™	Functions with gpuArray support (Statistics and Machine Learning Toolbox)	Analyze and Model Data on GPU (Statistics and Machine Learning Toolbox)
Image Processing Toolbox™	Functions with gpuArray support (Image Processing Toolbox)	GPU Computing (Image Processing Toolbox)
Medical Imaging Toolbox™	Functions with gpuArray support (Medical Imaging Toolbox)
Deep Learning Toolbox™	Functions with gpuArray support (Deep Learning Toolbox) See also Deep Learning with GPUs)	Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud (Deep Learning Toolbox) Deep Learning with MATLAB on Multiple GPUs (Deep Learning Toolbox)
Computer Vision Toolbox™	Functions with gpuArray support (Computer Vision Toolbox)	GPU Code Generation and Acceleration (Computer Vision Toolbox)
Communications Toolbox™	Functions with gpuArray support (Communications Toolbox)	Acceleration (Communications Toolbox)
5G Toolbox™	Functions with gpuArray support (5G Toolbox)
Antenna Toolbox™	Functions with gpuArray Support (Antenna Toolbox)
Signal Processing Toolbox™	Functions with gpuArray support (Signal Processing Toolbox)	Code Generation and GPU Support (Signal Processing Toolbox)
Audio Toolbox™	Functions with gpuArray support (Audio Toolbox)	Code Generation and GPU Support (Audio Toolbox)
Wavelet Toolbox™	Functions with gpuArray support (Wavelet Toolbox)	Code Generation and GPU Support (Wavelet Toolbox)
Predictive Maintenance Toolbox™	Functions with gpuArray support (Predictive Maintenance Toolbox)
Curve Fitting Toolbox™	Functions with gpuArray support (Curve Fitting Toolbox)
Radar Toolbox	Functions with gpuArray support (Radar Toolbox)

For a list of functions with gpuArray support in all MathWorks^® products, see gpuArray-supported functions. Alternatively, you can filter by product. On the Help bar, click Functions. In the function list, browse the left pane to select a product, for example, MATLAB. At the bottom of the left pane, select GPU Arrays. If you select a product that does not have gpuArray-enabled functions, then the GPU Arrays filter is not available.

Examples

Transfer Data to and from the GPU

To transfer data from the CPU to the GPU, use the gpuArray function.

Create an array X.

X = [1,2,3];

Transfer X to the GPU.

G = gpuArray(X);

Check that the data is on the GPU.

isgpuarray(G)

ans = logical
   1

Calculate the element-wise square of the array G.

GSq = G.^2;

Transfer the result GSq back to the CPU.

XSq = gather(GSq)

XSq = 1×3

     1     4     9

Check that the data is not on the GPU.

isgpuarray(XSq)

ans = logical
   0

Create Data on the GPU Directly

You can create data directly on the GPU directly by using some MATLAB functions and specifying the option "gpuArray".

Create an array of random numbers directly on the GPU.

G = rand(1,3,"gpuArray")

G =

    0.3640    0.5421    0.6543

Check that the output is stored on the GPU.

isgpuarray(G)

ans = logical
   1

Use MATLAB Functions with the GPU

This example shows how to use gpuArray-enabled MATLAB functions to operate with gpuArray objects. You can check the properties of your GPU using the gpuDevice function.

gpuDevice

ans = 
  CUDADevice with properties:

                 Name: 'NVIDIA RTX A5000'
                Index: 1 (of 2)
    ComputeCapability: '8.6'
          DriverModel: 'TCC'
          TotalMemory: 25544294400 (25.54 GB)
      AvailableMemory: 24734105600 (24.73 GB)
      DeviceAvailable: true
       DeviceSelected: true

  Show all properties.

Create a row vector that repeats values from -15 to 15. To transfer it to the GPU and create a gpuArray object, use the gpuArray function.

X = [-15:15 0 -15:15 0 -15:15];
gpuX = gpuArray(X);
whos gpuX

  Name      Size            Bytes  Class       Attributes

  gpuX      1x95              760  gpuArray              

To operate with gpuArray objects, use any gpuArray-enabled MATLAB function. MATLAB automatically runs calculations on the GPU. For more information, see Run MATLAB Functions on a GPU. For example, use diag, expm, mod, round, abs, and fliplr together.

gpuE = expm(diag(gpuX,-1)) * expm(diag(gpuX,1));
gpuM = mod(round(abs(gpuE)),2);
gpuF = gpuM + fliplr(gpuM);

Plot the results.

imagesc(gpuF);
colormap(flip(gray));

If you need to transfer the data back from the GPU, use gather. Transferring data back to the CPU can be costly, and is generally not necessary unless you need to use your result with functions that do not support gpuArray.

result = gather(gpuF);
whos result

  Name         Size            Bytes  Class     Attributes

  result      96x96            73728  double              

In general, running code on the CPU and the GPU can produce different results due to numerical precision and algorithmic differences between the GPU and CPU. Answers from the CPU and GPU are both equally valid floating point approximations to the true analytical result, having been subjected to different roundoff behavior during computation. In this example, the results are integers and round eliminates the roundoff errors.

Perform Monte Carlo Integration Using gpuArray-Enabled Functions

This example shows how to use MATLAB functions and operators with gpuArray objects to compute the integral of a function by using the Monte Carlo integration method.

Define the number of points to sample. Sample points in the domain of the function, the interval [-1,1] in both x and y coordinates, by creating random points with the rand function. To create a random array directly on the GPU, use the rand function and specify "gpuArray". For more information, see Establish Arrays on a GPU.

n = 1e6;
x = 2*rand(n,1,"gpuArray")-1;
y = 2*rand(n,1,"gpuArray")-1;

Define the function to integrate, and use the Monte Carlo integration formula on it. This function approximates the value of $$\pi$$ by sampling points within the unit circle. Because the code uses gpuArray-enabled functions and operators on gpuArray objects, the computations automatically run on the GPU. You can perform binary operations such as element-wise multiplication using the same syntax that you use for MATLAB arrays. For more information about gpuArray-enabled functions, see Run MATLAB Functions on a GPU.

f = x.^2 + y.^2 <= 1;
result = 4*nnz(f)/n

result = 3.1433

Limitations

• None of the following can exceed intmax("int32"):

  • The number of elements of a dense array.

  • The number of nonzero elements of a sparse array.

  • The size in any given dimension. For example, zeros(0,3e9,"gpuArray") is not allowed.

• Distributing a gpuArray among workers in a parallel pool using the distributed or codistributed functions is not supported. If you have multiple GPUs and each worker in your parallel pool has access to a unique GPU, you can instead manually split or initially generate your data as multiple gpuArray objects on different workers. For examples showing how to use gpuArray data in a parallel pool, see Run MATLAB Functions on Multiple GPUs.

• Sparse gpuArray limitations:

  • Logical sparse GPU arrays are not supported.

  • Sparse GPU arrays only support referencing whole rows or columns by index.

  • Assigning values to sparse GPU arrays by index is not supported.

  For more information, see Work with Sparse Arrays on a GPU.

Tips

• If you need better performance, or if a function is not available on the GPU, gpuArray supports the following options:

  • To precompile and run purely element-wise code on gpuArray objects, use the arrayfun function.

  • To run C++ code containing CUDA^® device code or library calls, use a MEX function. For more information, see Run MEX Functions Containing CUDA Code.

  • To run existing GPU kernels written in CUDA C++, use the MATLAB CUDAKernel interface. For more information, see Run CUDA or PTX Code on GPU.

  • To generate CUDA code from MATLAB code, use GPU Coder™. For more information, see Get Started with GPU Coder (GPU Coder).

• Most GPUs perform calculations faster in single precision than in double precision. If your workflow does not require double precision, consider converting your data to single precision using the single function or creating single-precision gpuArray data directly. For more information, see Perform Calculations in Single Precision.

• To control the random number stream on the GPU, use the gpurng function.

Alternatives

You can also create a gpuArray object using some MATLAB functions by specifying a gpuArray output. The following table lists the MATLAB functions that enable you to create gpuArray objects directly. For more information, see the Extended Capabilities section of the function reference page.

eye(___,"gpuArray")	createArray(___,"gpuArray") (since R2024a)
false(___,"gpuArray")	gpuArray.colon
Inf(___,"gpuArray")	gpuArray.freqspace
NaN(___,"gpuArray")	gpuArray.linspace
ones(___,"gpuArray")	gpuArray.logspace
rand(___,"gpuArray")	gpuArray.speye
randi(___,"gpuArray")	gpuArray.sprand
randn(___,"gpuArray")	gpuArray.sprandn
true(___,"gpuArray")	gpuArray.sprandsym
zeros(___,"gpuArray")

Extended Capabilities

Thread-Based Environment

Run code in the background using MATLAB® backgroundPool or accelerate code with Parallel Computing Toolbox™ ThreadPool.

This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.

Version History

Introduced in R2010b

R2025a: Reduce memory usage with single-precision sparse GPU arrays

You can now create and use single-precision sparse GPU arrays. Sparse matrices provide efficient storage of data that has a large percentage of zeros and reduce computation time by eliminating operations on zero elements. Using single-precision sparse GPU arrays allows you to reduce memory usage further and accelerate calculations by taking advantage of your GPU's single-precision floating-point units (FPUs).

To convert a gpuArray to single precision, use the single function.

For more information, see Work with Sparse Arrays on a GPU.