parallel.Cluster
Access cluster properties and behaviors
Container Hierarchy
Parent | None |
Children | parallel.Job, parallel.Pool |
Description
A parallel.Cluster object provides access to a cluster, which controls the job queue, and distributes tasks to workers for execution.
Types
The two categories of clusters are the MATLAB® Job Scheduler and common job scheduler (CJS). The MATLAB Job Scheduler is available in the MATLAB Parallel Server™. The CJS clusters encompass all other types, including the local, generic, and third-party schedulers.
Use MJSComputeCloud
objects to interact with MATLAB
Parallel Server for Amazon EC2® clusters.
The following table describes the available types of cluster objects.
Cluster Type | Description |
---|---|
parallel.cluster.MJS | Interact with MATLAB Job Scheduler cluster on-premises |
parallel.cluster.MJSComputeCloud | Interact with MATLAB Parallel Server for Amazon EC2 cluster |
parallel.cluster.Local | Interact with CJS cluster running locally on client machine |
parallel.cluster.HPCServer | Interact with CJS cluster running Windows Microsoft HPC Server |
parallel.cluster.LSF | Interact with CJS cluster running LSF |
parallel.cluster.PBSPro | Interact with CJS cluster running Altair PBS Pro or OpenPBS |
parallel.cluster.Torque | Interact with CJS cluster running TORQUE |
parallel.cluster.Slurm | Interact with CJS cluster running Slurm |
parallel.cluster.Generic | Interact with CJS cluster using the generic interface |
Methods
Common to All Cluster Types
batch | Run MATLAB script or function on worker |
createCommunicatingJob | Create communicating job on cluster |
createJob | Create independent job on cluster |
findJob | Find job objects stored in cluster |
isequal | True if clusters have same property values |
parpool | Create parallel pool on cluster |
saveAsProfile | Save cluster properties to specified profile |
saveProfile | Save modified cluster properties to its current profile |
MATLAB Job Scheduler
changePassword | Prompt user to change MATLAB Job Scheduler password |
demote | Demote job in cluster queue |
logout | Log out of MATLAB Job Scheduler cluster |
promote | Promote job in MATLAB Job Scheduler cluster queue |
resume | Resume processing queue in MATLAB Job Scheduler |
MJSComputeCloud
shutdown | Shut down cloud cluster |
start | Start cloud cluster |
wait (cluster) | Wait for cloud cluster to change state |
HPC Server, PBS Pro, LSF, TORQUE, Slurm, and Local Clusters
getDebugLog | Read output messages from job run in CJS cluster |
Generic
getDebugLog | Read output messages from job run in CJS cluster |
getJobClusterData | Get specific user data for job on generic cluster |
getJobFolder | Folder on client where jobs are stored |
getJobFolderOnCluster | Folder on cluster where jobs are stored |
getLogLocation | Log location for job or task |
setJobClusterData | Set specific user data for job on generic cluster |
Properties
Common to All Cluster Types
The following properties are common to all cluster object types.
Property | Description |
---|---|
ClusterMatlabRoot | Specifies path to MATLAB for workers to use |
Host | Host name of the cluster head node |
JobStorageLocation | Location where cluster stores job and task information |
Jobs | List of jobs contained in this cluster |
LicenseNumber | License number to use when running jobs with this cluster |
Modified | True if any properties in this cluster have been modified |
NumThreads | Number of computational threads for workers |
NumWorkers | Number of workers available for this cluster |
OperatingSystem | Operating system of nodes used by cluster |
Profile | Profile used to build this cluster |
RequiresOnlineLicensing | True if the cluster is using online licensing |
Type | Type of this cluster |
UserData | Information associated with cluster object within client session |
Specify the JobStorageLocation
property as one of the following.
If you use the generic scheduler interface in remote mode to interact with a third-party scheduler, and the client and workers use different operating systems, specify a structure. The structure must have the fields
'windows'
and'unix'
. The fields are the Windows and Unix path corresponding to the folder where the cluster stores job and task information. The following structure specifies the same folder using a Windows UNC path ('\\organization\some\path'
) and a Unix path ('/organization/some/path'
):struct('windows', '\\organization\some\path', 'unix', '/organization/some/path')
Otherwise, use a character vector or string to specify the folder where the cluster stores job and task information.
MATLAB Job Scheduler
MJS
cluster objects have the following properties in
addition to the common properties:
Property | Description |
---|---|
AllHostAddresses | IP addresses of the cluster host |
BusyWorkers | Workers currently running tasks |
IdleWorkers | Workers currently available for running tasks |
HasSecureCommunication | True if cluster is using secure communication |
MaxNumWorkers | Maximum number of workers this cluster can have.
|
Name | Name of this cluster |
NumBusyWorkers | Number of workers currently running tasks |
NumIdleWorkers | Number of workers available for running tasks |
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
PromptForPassword | True if system should prompt for password when authenticating user |
SecurityLevel | Degree of security applied to cluster and its jobs. For descriptions of security levels, see Set MATLAB Job Scheduler Cluster Security (MATLAB Parallel Server). |
State | Current state of cluster |
Username | User accessing cluster |
MJSComputeCloud
MJSComputeCloud
cluster objects have the following
properties in addition to the common properties:
Property | Description |
---|---|
BusyWorkers | Workers currently running tasks |
Certificate | Cluster SSL certificate |
HasSecureCommunication | True if cluster is using secure communication |
Identifier | Unique cluster identifier |
IdleWorkers | Workers currently available for running tasks |
MatlabVersion | Version of MATLAB running on the workers |
MaxNumWorkers | Maximum number of workers this cluster can have.
|
Name | Name of this cluster |
NumBusyWorkers | Number of workers currently running tasks |
NumIdleWorkers | Number of workers available for running tasks |
NumWorkersRequested | Number of workers requested for this cluster. The cluster adds or removes workers as soon as possible to reach this number. |
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
SharedState | The shared state of the cluster, which can be:
|
ShutdownAt | Shutdown time or event |
State | Current state of cluster |
Username | User accessing cluster |
Local
Local
cluster objects have no editable properties beyond
the properties common to all clusters.
HPC Server
HPCServer
cluster objects are supported on clients running
Windows.
HPCServer
cluster objects have the following properties in
addition to the common properties:
Property | Description |
---|---|
ClusterVersion | Version of Microsoft Windows HPC Server running on the cluster |
HasSharedFilesystem | Specify whether client and cluster nodes share JobStorageLocation |
JobDescriptionFile | Name of XML job description file to use when creating jobs |
JobTemplate | Name of job template to use for jobs submitted to HPC Server |
Name | Name of this cluster |
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
UseSOAJobSubmission | Allow service-oriented architecture (SOA) submission on HPC Server |
PBS Pro and TORQUE
PBSPro
cluster objects are supported on clients running
Windows or Linux. Torque
cluster objects are supported on
clients running Linux.
PBSPro
and Torque
cluster objects have
the following properties in addition to the common properties:
Property | Description |
---|---|
CommunicatingJobWrapper | Script that cluster runs to start workers |
RcpCommand | Command to copy files to and from client |
ResourceTemplate | Specify qsub options to request resources
during job submission |
RshCommand | Remote execution command used on worker nodes during communicating job |
HasSharedFilesystem | Specify whether client and cluster nodes share JobStorageLocation |
ProcsPerNode |
Number of processors per node, specified as a finite positive integer scalar. When you submit a job to the cluster, the number of cores per node that MATLAB requests is guaranteed to be less than or equal to
MATLAB requests the smallest number of cores per node required to run the job.
|
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
SubmitArguments | Specify additional arguments to use when submitting jobs |
LSF
LSF
cluster objects are supported on clients running
Windows, macOS, or Linux.
LSF
cluster objects have the following properties in
addition to the common properties:
Property | Description |
---|---|
ClusterName | Name of LSF cluster |
CommunicatingJobWrapper | Script that the cluster runs to start workers |
HasSharedFilesystem | Specify whether client and cluster nodes share JobStorageLocation |
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
ResourceTemplate | Specify bsub options to request resources
during job submission |
SubmitArguments | Specify additional arguments to use when submitting jobs |
Slurm
Slurm
cluster objects are supported on clients running
Linux.
Slurm
cluster objects have the following properties in
addition to the common properties:
Property | Description |
---|---|
ClusterName | Name of the Slurm cluster |
CommunicatingJobWrapper | Script that the cluster runs to start workers |
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
ResourceTemplate | Specify sbatch options to request
resources during job submission |
SubmitArguments | Specify additional arguments to use when submitting jobs |
Generic
Generic
cluster objects are supported on clients running
Windows, macOS, or Linux.
If you create a generic cluster object from an R2017a or later profile, you have the following properties in addition to the common properties:
Property | Description |
---|---|
AdditionalProperties | Additional properties for plugin scripts |
HasSharedFilesystem | Specify whether client and cluster nodes share JobStorageLocation |
PreferredPoolNumWorkers | Preferred number of workers to start in a parallel pool |
PluginScriptsLocation | Folder containing scheduler plugin scripts |
If you create a generic cluster object from an R2016b or earlier profile, you have the following properties in addition to the common properties:
Property | Description |
---|---|
CancelJobFcn | Function to run when cancelling job |
CancelTaskFcn | Function to run when cancelling task |
CommunicatingSubmitFcn | Function to run when submitting communicating job |
DeleteJobFcn
| Function to run when deleting job |
DeleteTaskFcn | Function to run when deleting task |
GetJobStateFcn | Function to run when querying job state |
HasSharedFilesystem | Specify whether client and cluster nodes share JobStorageLocation |
IndependentSubmitFcn | Function to run when submitting independent job |
Help
For further help on cluster objects, including links to help for specific cluster types and object properties, type:
help parallel.Cluster
Version History
Introduced in R2012a