Cette page a été traduite par traduction automatique. Cliquez ici pour voir la dernière version en anglais.

Tableaux distribués

Analysez des ensembles de big data en parallèle à l'aide de tableaux distribués et d'une exécution simultanée

Parallel Computing Toolbox™ prend en charge les tableaux distribués pour partitionner de grands tableaux sur plusieurs workers MATLAB^®. Vous opérez sur l'ensemble du tableau en tant qu'entité unique. Cependant, les workers n'opèrent que sur leur partie du tableau et transfèrent automatiquement les données entre eux si nécessaire. L'exécution simultanée est prise en charge par la construction de langage à programme unique et données multiples (spmd) pour faciliter la communication entre les workers. Utilisez des opérations et des fonctions matricielles distribuées pour travailler directement avec ces tableaux sans autre modification. Vous pouvez utiliser des tableaux distribués dans Parallel Computing Toolbox pour exécuter des applications Big Data en utilisant la mémoire combinée de votre cluster.

Fonctions

développer tout

Fonctions clés

`distributed`	Create and access elements of distributed arrays from client
`gather`	Transfer distributed array, `Composite` object, or `gpuArray` object to local workspace
`spmd`	Exécuter du code en parallèle sur les workers du pool parallèle
`Composite`	Create and access nondistributed variables on multiple workers from client
`parallel.pool.Constant`	Build and use constant from data or function handle
`redistribute`	Redistribute codistributed array with another distribution scheme
`codistributed`	Access elements of arrays distributed among workers in parallel pool
`codistributor1d`	1-D distribution scheme for codistributed array
`codistributor2dbc`	2-D block-cyclic distribution scheme for codistributed array
`codistributed.build`	Create codistributed array from distributed data
`for`	`for`-boucle sur une plage distribuée
`getLocalPart`	Local portion of codistributed array
`globalIndices`	Global indices for local part of codistributed array
`spmdReduce`	Reduce arrays on `spmd` workers (depuis R2022b)
`write`	Write distributed data to an output location
`pagefun`	Apply function to each page of distributed or GPU array

Classes

développer tout

Classes importantes

parallel.Pool Pool parallèle de workers

Exemples et procédures

Create and Use Distributed Arrays
When your data array is too big to fit into the memory of a single machine, you can create a distributed array.
Exécuter des fonctions MATLAB avec des tableaux distribués
Fonctions MATLAB qui fonctionnent sur des tableaux distribués.
Distributing Arrays to Parallel Workers
Use datastore or distributed to create distributed arrays and partition the data among your workers.
Run Single Programs on Multiple Data Sets
Use spmd statements to run the same code on multiple datasets and control codistributed arrays.
Access Worker Variables with Composites
Composite objects in the MATLAB client session let you directly access data values on the workers.
Train Network in Parallel with Custom Training Loop (Deep Learning Toolbox)
This example shows how to set up a custom training loop to train a network in parallel.
Use spmdReduce to Achieve MPI_Allreduce Functionality
In this example, we look at the spmdReduce function and the functions that build on it: spmdPlus and spmdCat.
Numerical Estimation of Pi Using Message Passing
This example shows the basics of working with spmd statements, and how they provide an interactive means of performing parallel computations.
Choose Between spmd, parfor, and parfeval
Compare and contrast spmd against other parallel computing functionality such as parfor and parfeval.

Concepts

Nondistributed Versus Distributed Arrays
Describes the various types of arrays used in communicating jobs.
Working with Codistributed Arrays
Describes how to use codistributed arrays for calculation.
Looping Over a Distributed Range (for-drange)
Describes how to program a for-loop with codistributed arrays.
Work with Remote Data
Work with remote data in Amazon S3™, Azure^® Blob Storage, or HDFS™.

Exemples présentés

Use Distributed Arrays to Solve Systems of Linear Equations with Direct Methods

Solve a system of linear equations of the form $A x = b$ in parallel with a direct method using distributed arrays.

Ouvrir le live script

Use Distributed Arrays to Solve Systems of Linear Equations with Iterative Methods

For large-scale mathematical computations, iterative methods can be more efficient than direct methods. This example shows how you can solve systems of linear equations of the form $A x = b$ in parallel using distributed arrays with iterative methods.

Ouvrir le live script

Solve Differential Equation Using Multigrid Preconditioner on Distributed Discretization

Solve Poisson's equation using a preconditioned iterative solver and distributed arrays. By using distributed arrays, you can scale up the calculation using the memory of a cluster of machines, not just the memory of a single machine. You can scale up without changing your code.

Ouvrir le live script