parallel.Cluster

Access cluster properties and behaviors

Constructors

parcluster

getCurrentCluster (in the workspace of the MATLAB worker)

Container Hierarchy

Parent	None
Children	parallel.Job, parallel.Pool

Description

A parallel.Cluster object provides access to a cluster, which controls the job queue, and distributes tasks to workers for execution.

Types

The two categories of clusters are the MATLAB^® Job Scheduler and common job scheduler (CJS). The MATLAB Job Scheduler is available in the MATLAB Parallel Server™. The CJS clusters encompass all other types, including the local, generic, and third-party schedulers.

Use MJSComputeCloud objects to interact with MATLAB Parallel Server for Amazon EC2^® clusters.

The following table describes the available types of cluster objects.

Cluster Type	Description
parallel.cluster.MJS	Interact with MATLAB Job Scheduler cluster on-premises
parallel.cluster.MJSComputeCloud	Interact with MATLAB Parallel Server for Amazon EC2 cluster
parallel.cluster.Local	Interact with CJS cluster running locally on client machine
parallel.cluster.HPCServer	Interact with CJS cluster running Windows Microsoft HPC Server
parallel.cluster.LSF	Interact with CJS cluster running LSF
parallel.cluster.PBSPro	Interact with CJS cluster running Altair PBS Pro or OpenPBS
parallel.cluster.Torque	Interact with CJS cluster running TORQUE
parallel.cluster.Slurm	Interact with CJS cluster running Slurm
parallel.cluster.Generic	Interact with CJS cluster using the generic interface

Methods

Common to All Cluster Types

batch	Run MATLAB script or function on worker
createCommunicatingJob	Create communicating job on cluster
createJob	Create independent job on cluster
findJob	Find job objects stored in cluster
isequal	True if clusters have same property values
parpool	Create parallel pool on cluster
saveAsProfile	Save cluster properties to specified profile
saveProfile	Save modified cluster properties to its current profile

MATLAB Job Scheduler

changePassword	Prompt user to change MATLAB Job Scheduler password
demote	Demote job in cluster queue
logout	Log out of MATLAB Job Scheduler cluster
promote	Promote job in MATLAB Job Scheduler cluster queue
resume	Resume processing queue in MATLAB Job Scheduler

MJSComputeCloud

shutdown	Shut down cloud cluster
start	Start cloud cluster
wait (cluster)	Wait for cloud cluster to change state

HPC Server, PBS Pro, LSF, TORQUE, Slurm, and Local Clusters

getDebugLog

Read output messages from job run in CJS cluster

Generic

getDebugLog	Read output messages from job run in CJS cluster
getJobClusterData	Get specific user data for job on generic cluster
getJobFolder	Folder on client where jobs are stored
getJobFolderOnCluster	Folder on cluster where jobs are stored
getLogLocation	Log location for job or task
setJobClusterData	Set specific user data for job on generic cluster

Properties

Common to All Cluster Types

The following properties are common to all cluster object types.

Property	Description
ClusterMatlabRoot	Specifies path to MATLAB for workers to use
Host	Host name of the cluster head node
JobStorageLocation	Location where cluster stores job and task information
Jobs	List of jobs contained in this cluster
LicenseNumber	License number to use when running jobs with this cluster
Modified	True if any properties in this cluster have been modified
NumThreads	Number of computational threads for workers
NumWorkers	Number of workers available for this cluster
OperatingSystem	Operating system of nodes used by cluster
Profile	Profile used to build this cluster
RequiresOnlineLicensing	True if the cluster is using online licensing
Type	Type of this cluster
UserData	Information associated with cluster object within client session

Specify the JobStorageLocation property as one of the following.

• If you use the generic scheduler interface in remote mode to interact with a third-party scheduler, and the client and workers use different operating systems, specify a structure. The structure must have the fields 'windows' and 'unix'. The fields are the Windows and Unix path corresponding to the folder where the cluster stores job and task information. The following structure specifies the same folder using a Windows UNC path ('\\organization\some\path') and a Unix path ('/organization/some/path'):

  struct('windows', '\\organization\some\path', 'unix', '/organization/some/path')

• Otherwise, use a character vector or string to specify the folder where the cluster stores job and task information.

MATLAB Job Scheduler

MJS cluster objects have the following properties in addition to the common properties:

Property	Description
AllHostAddresses	IP addresses of the cluster host
BusyWorkers	Workers currently running tasks
IdleWorkers	Workers currently available for running tasks
HasSecureCommunication	True if cluster is using secure communication
MaxNumWorkers	Maximum number of workers this cluster can have. When you use a cluster without automatic resizing, this value is the number of workers in the cluster. When you use a cluster with automatic resizing, workers are added automatically up to this maximum value as necessary. For more information on automatic resizing, see Set up MATLAB Job Scheduler Cluster for Auto-Resizing (MATLAB Parallel Server).
Name	Name of this cluster
NumBusyWorkers	Number of workers currently running tasks
NumIdleWorkers	Number of workers available for running tasks
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
PromptForPassword	True if system should prompt for password when authenticating user
SecurityLevel	Degree of security applied to cluster and its jobs. For descriptions of security levels, see Set MATLAB Job Scheduler Cluster Security (MATLAB Parallel Server).
State	Current state of cluster
Username	User accessing cluster

MJSComputeCloud

MJSComputeCloud cluster objects have the following properties in addition to the common properties:

Property	Description
BusyWorkers	Workers currently running tasks
Certificate	Cluster SSL certificate
HasSecureCommunication	True if cluster is using secure communication
Identifier	Unique cluster identifier
IdleWorkers	Workers currently available for running tasks
MatlabVersion	Version of MATLAB running on the workers
MaxNumWorkers	Maximum number of workers this cluster can have. When you use a cluster without automatic resizing, this value is the number of workers when you started the cluster. When you use a cluster with automatic resizing, workers are added automatically up to this maximum value as necessary. For more information on automatic resizing, see Resize Clusters Automatically.
Name	Name of this cluster
NumBusyWorkers	Number of workers currently running tasks
NumIdleWorkers	Number of workers available for running tasks
NumWorkersRequested	Number of workers requested for this cluster. The cluster adds or removes workers as soon as possible to reach this number.
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
SharedState	The shared state of the cluster, which can be: Personal – Only you can use this cluster, so long as you created it. Shareable – Anyone can use this cluster.
ShutdownAt	Shutdown time or event
State	Current state of cluster
Username	User accessing cluster

Local

Local cluster objects have no editable properties beyond the properties common to all clusters.

HPC Server

HPCServer cluster objects are supported on clients running Windows.

HPCServer cluster objects have the following properties in addition to the common properties:

Property	Description
ClusterVersion	Version of Microsoft Windows HPC Server running on the cluster
HasSharedFilesystem	Specify whether client and cluster nodes share JobStorageLocation
JobDescriptionFile	Name of XML job description file to use when creating jobs
JobTemplate	Name of job template to use for jobs submitted to HPC Server
Name	Name of this cluster
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
UseSOAJobSubmission	Allow service-oriented architecture (SOA) submission on HPC Server

PBS Pro and TORQUE

PBSPro cluster objects are supported on clients running Windows or Linux. Torque cluster objects are supported on clients running Linux.

PBSPro and Torque cluster objects have the following properties in addition to the common properties:

Property	Description
CommunicatingJobWrapper	Script that cluster runs to start workers
RcpCommand	Command to copy files to and from client
ResourceTemplate	Specify qsub options to request resources during job submission
RshCommand	Remote execution command used on worker nodes during communicating job
HasSharedFilesystem	Specify whether client and cluster nodes share JobStorageLocation
ProcsPerNode	Number of processors per node, specified as a finite positive integer scalar. When you submit a job to the cluster, the number of cores per node that MATLAB requests is guaranteed to be less than or equal to ProcsPerNode. Set ProcsPerNode equal to the maximum number of processors you want MATLAB to request from each cluster node. MATLAB requests the smallest number of cores per node required to run the job. If the NumThreads property of the cluster is less than or equal to ProcsPerNode, MATLAB requests NumThreads processors per worker, then maximizes the number of workers per node. For example if ProcsPerNode is 16 and NumThreads is 5, MATLAB requests 15 cores, the largest multiple of 5 that is less than ProcsPerNode. If NumThreads of the cluster is greater than ProcsPerNode, MATLAB requests ProcsPerNode processors per node. When NumThreads is greater than ProcsPerNode, you might encounter performance issues. As a best practice, set NumThreads less than or equal to ProcsPerNode. For more information, see Edit Number of Workers and Cluster Settings.
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
SubmitArguments	Specify additional arguments to use when submitting jobs

LSF

LSF cluster objects are supported on clients running Windows, macOS, or Linux.

LSF cluster objects have the following properties in addition to the common properties:

Property	Description
ClusterName	Name of LSF cluster
CommunicatingJobWrapper	Script that the cluster runs to start workers
HasSharedFilesystem	Specify whether client and cluster nodes share JobStorageLocation
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
ResourceTemplate	Specify bsub options to request resources during job submission
SubmitArguments	Specify additional arguments to use when submitting jobs

Slurm

Slurm cluster objects are supported on clients running Linux.

Slurm cluster objects have the following properties in addition to the common properties:

Property	Description
ClusterName	Name of the Slurm cluster
CommunicatingJobWrapper	Script that the cluster runs to start workers
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
ResourceTemplate	Specify sbatch options to request resources during job submission
SubmitArguments	Specify additional arguments to use when submitting jobs

Generic

Generic cluster objects are supported on clients running Windows, macOS, or Linux.

If you create a generic cluster object from an R2017a or later profile, you have the following properties in addition to the common properties:

Property	Description
AdditionalProperties	Additional properties for plugin scripts
HasSharedFilesystem	Specify whether client and cluster nodes share JobStorageLocation
PreferredPoolNumWorkers	Preferred number of workers to start in a parallel pool
PluginScriptsLocation	Folder containing scheduler plugin scripts

If you create a generic cluster object from an R2016b or earlier profile, you have the following properties in addition to the common properties:

Property	Description
CancelJobFcn	Function to run when cancelling job
CancelTaskFcn	Function to run when cancelling task
CommunicatingSubmitFcn	Function to run when submitting communicating job
DeleteJobFcn	Function to run when deleting job
DeleteTaskFcn	Function to run when deleting task
GetJobStateFcn	Function to run when querying job state
HasSharedFilesystem	Specify whether client and cluster nodes share JobStorageLocation
IndependentSubmitFcn	Function to run when submitting independent job

Help

For further help on cluster objects, including links to help for specific cluster types and object properties, type:

help parallel.Cluster