How to use all CPU cores

I have 16 cores and 32 logical processors but only 16 out of the 32 processors are being used for my Matlab which takes very long run time. How can I make my computer to utilize all of the CPU and processors?

Answer 1

[Aashita Dutta]

As per my understanding, you are trying to fully utilize all CPU logical processors for faster execution of the program.

To resolve this issue, you can leverage ‘Parallel Computing Toolbox’ that accelerates workflow using multicore processors where ‘Parallel for-Loops’ (Section 2) can help execute loops parallelly. Also ‘Profiling’ is helpful to investigate parts of code where the most time is spent.

For more information, please refer to the ‘Parallel Computing’ document.

Answer 2

[Walter Roberson]

Each pair of "logical" cores shares physical core resources. "Logical" cores have their own set of registers, but only one set of the registers are active at any one time, with the ability to quickly switch between register sets. But there are only one set of arithmetic and logical units.

The situation is sort of like having a truck with two drivers but only one of the drivers has control at any one time, with it being fast to switch between drivers.

The advantage of logical cores is that the alternate control can kick in quickly during a period that would normally be down time -- such as a computation task having ended, or waiting for an interruption to happen. Instead of the logic units going idle when a thread is waiting for something to happen, control is switched to the other logical core which makes productive use of the resources when they would otherwise be idle.

Logical cores do not have additional computation resources. You cannot have computation on both logical cores at the same time: when one member of the logical core is busy computing, the other member of the logical core set is either suspended waiting to regain control or else is waiting on some kind of external event.

When you have long-running computations, there is little to no idle time (occasionally you might wait for a memory page to become available), so when you have long-running computations, the logical cores end up fighting for control... which is unproductive.

As well, because logical cores keep the physical core busier, the physical core has fewer chances to cool down, so running logical cores tends to keep the CPUs hotter. It is common for the CPU to automatically reduce the clock speed to reduce the heat. As a result, when logical cores are used, the CPUs tend to slow down -- reducing the amount of computation to below what would be possible if logical cores are not used.

This thermal regulation process depends on the CPU architecture. My understanding is that AMD Ryzen processors do not slow down due to thermal regulation when logical cores are used, but Intel processors do slow down when logical cores are used. Intel processors tend to be a bit faster than corresponding AMD processors, with AMD processors typically having the advtange of having many more physical cores. Intel thus tends to have the advantage for processes that are effectively single-threaded, but AMD tends to have the advantage for multi-threaded processes.

Anyhow... whether logical cores have any advantage for you depends on how much your process tends to wait for something to happen. If your workers are reading from disk or are writing I/O or are waiting on external devices, then logical cores potentially have value; if your workers are CPU bound, then logical cores tend to be slower than not using logical cores.

Answer 3

[Gabriel Anton]

MATLAB's parallel computing capabilities depend on how the Parallel Computing Toolbox is configured and how MATLAB interacts with your system's CPU. Since you have 16 physical cores and 32 logical processors (likely due to Hyper-Threading or SMT), MATLAB might not be fully utilizing all available resources by default. Here’s how you can improve CPU utilization:

[Walter Roberson]

You accidentally left out the instructions.

By default, MATLAB configures all physical cores in the pool. By default the pool size is the same as the number of cores.

This configuration is the best one when each of the workers is CPU bound and happen to work well in single threaded mode.

If tasks are CPU bound and happen to be mostly operations on larger matrices, especially linear algebra calculations, then there can be benefits to reconfiguring the pool to have fewer members each with multiple physical cores.

If the tasks happen to contain notable waits on outside events, such as reading from disk or waiting for interruption, then sometimes there is benefit to enabling logical cores.

On the other hand, if the tasks do involve notable waiting for disk, then there is the problem that disk controllers are typically limited to transferring a small number of channels simultaneously, and disk drives are almost always limited reading only one file at a time. Even drives that do good optimization of several outstanding read requests are almost always limited to reading one stream of blocks at a time, so when you have more than roughly 3 outstanding read requests at a time, performance begins to suffer. To get around this problem you need to carefully plan the placement of data on separate drives and separate drive controllers to optimize drive performance. The maximum practical performance is typically with four drives distributed across two controllers and two or three outstanding read requests per drive. In short even with good planning, it is tricky to optimize I/O on more than roughly 8 workers. (Not impossible, if you are using server class machines that are well equipped and very carefully implemented.)

All of which is to say that a lot of the time, the default of one worker per physical core works out the best, whereas enabling logical cores tends to slow things down.