Does MATLAB use all cores by default when running a program?

I have been thinking about purchasing MATLAB's Parallel Computing toolbox because I will likely be responsible for some large scale computation on a single desktop computer but before that I would like to know one thing. So I guess parallel computation especially MATLAB's related toolbox can utilize all cores available in a computer to speed computation. My question is, since almost all modern PCs contain multiple cores by default, what does the normal MATLAB do when running a program? Is it only using one of those cores when parallel computation toolbox is not being used?

Answer 1

[Adam]

User-written code will only use one core in base Matlab. I don't really know enough about how builtin functions are implemented, but many of those are heavily optimised so I imagine that a number of builtin functions do use multiple CPU cores if they are available (though I may be wrong in that - either way though, builtin functions are in many cases heavily optimised, by whatever method).

For your own code you do need the Parallel Computing Toolbox to utilise multiple cores (and GPU cores) and you can set the number of cores it uses in the settings. For example, I have 28 physical cores (56 with hyper-threading, but I don't really know what that means, Matlab only sees the 28 physical cores anyway) and if the computation is sufficiently intensive it will use all of them during parallel processing.

Edit: Looks like builtin functions do use multithreading:

https://uk.mathworks.com/discovery/matlab-multicore.html

[Imam]

Thanks for your answer, I think I will first get its free trial and see how it affects the computation speed.

[Adam]

Yes, it is definitely worth getting the trial. You will need to get used to certain ways of programming to get the best out of it too which you can learn during the trial. I only use it occasionally so each time I have to try to remember everything to ensure I get the best out of parallelisation. Bad parallelisation can slow down your code by orders of magnitude so make sure you are happy that you have got the best out of it when measuring its effect on speed.

[Ram Kumar Venkateswaran]

Hi Adam, What do you mean by bad parallelisms

[Walter Roberson]

(Dial) (Ring, Ring, Ring) Hi, Sally, grab a piece of paper and a pen. Yes, I'll wait. Got them? Okay, write down these two numbers: 10, and 22. Got them? Okay, I want you to add those two numbers. I'll call you back later to ask for the answer. Bye!"

(Dial) (Ring) "Bob? Grab a pen and paper. You have them already? Great! Write down these two numbers. 11 and 23. I want you to add them. I'll call you back later for the answer. Bye!"

(Dial) (Ring) "Hello Sally, are you done that work yet? You are? Good. What was the result? 32? Good. Thanks, Bye."

(Dial) (Ring, Ring, Ring, Ring, Ring) "Bob? Are you done the work yet? No? The house-keeper took away your messy piece of paper and is rewriting it all to be more legible and easier to find everything? Another 17 minutes, you say? Yes, I'll hold until then."

[Walter Roberson]

Suppose there was a fixed overhead of 1/100 second to communicate with another process, sending it instructions or getting a response from it. Suppose that you had 4 parallel processes. Then instructing the work would take 4/100 seconds, and getting the results would take 4/100 seconds, for a net of 8/100 seconds. Suppose each CPU can do 10^9 instructions per second.

Then if you had just gone ahead and done the work yourself, after 8/100 seconds you would have been able to do 8/10^7 instructions.

How much work would the 4 parallel processes be able to do in that time?

• t = 0 to t = 1/100: instruct P1
• t = 1/100 to t = 2/100: instruct P2. P1 progresses 10^9/100 = 10^7 instructions, for a total of 10^7 so far
• t = 2/100 to t = 3/100: instruct P3. P1 progresses 10^7 (total 2E7), P2 progresses 10^7 (total 1E7)
• t = 3/100 to t = 4/100: instruct P4. P1 progresses 10^7 (total 3E7), P2 progresses 10^7 (total 2E7), P3 progresses 10^7 (total 1E7)
• t = 4/100 to t = 5/100: query P1. It solved anything up to 3E7. P2 progresses 1E7 (total 3E7), P3 progresses 10^7 (total 2E7), P4 progresses 10^7 (total 1E7)
• t = 5/100 to t = 6/100: query P2. It solved up to 3E7. P3 progresses to 3E7, P4 progresses to 2E7
• t = 6/100 to t = 7/100: query P3. It solved up to 3E7. P4 progresses to 3E7
• t = 7/100 to t = 8/100: query P4, It solved up to 3E7

Total work done by children processes: up to 3E7 * 4 = 12E7

which is actually a net win, provided that the size of the problem to be done was about 3E7. More generally, with N processes (and the coordinator is somehow "free"), you get N*(N-1) worth of work in 2*N periods, and the efficiency ratio becomes (N-1)/2 : 1. If the problem cost is too small then you are further behind in using parallel processing.

Answer 2

[Jan]

Many builtin functions use multithreading, e.g. sum, filter and linear algebra functions. They use multiple cores only above a certain limit of input data size. If this happens in your code, the parallel processing toolbox cannot accelerate the code sufficiently, because all cores are working already. But e.g. if the processor idles because it is waiting for data from the disk, a multi-threaded code can be much faster.

[Imam]

The particular task that I am going to get my hands on is analyzing thousands of images (matrices) with dimension 512x512 at least. Using serial/standard for-loops to modify those matrices one at a time seems to be too slow that's why I consider using parallel algorithm.

[Greg Chance]

Hi, can you please elaborate on your answer @Jan?

1) Specifically does the variance function use multi-threading?

2) And can you say what the input data size needs to be before it does mulithreading?

3) Is the answer limited by input precision (similar to the numpy issue here) and if so how to control?

Thanks!

[Steven Lord]

1) The var function is a MATLAB function file, so it doesn't use multiple threads directly. Some of the built-in functions called in that function file, like sum, are multi-threaded if the problem size is large enough. The list in this Answers post is a bit old but generally speaking each release we try to improve the performance of our functions and one of the ways we can do that is to thread the function.

2) No. Do not try to tailor your code to our internal thresholds that can change in any release without warning. If you run a piece of code and it's not performing as well as you expect, and a built-in function is the bottleneck (as shown by the profiling tools in MATLAB, report that to Technical Support and ask if there's an alternative or if they can report that to the development staff for investigation.

3) If you pass single precision data into var, the result you receive back will be in single precision. The default behavior of sum (which is the behavior used in var) is "If X is floating point, that is double or single, S has the same class as X. If X is not floating point, S has class double." That second sentence doesn't apply in this context, since var throws an error if you try to call it with integer data.

[Ram Kumar Venkateswaran]

Hi Jan, But when I have a parallel computing toolbox, I can manually give the number of cores right? Is there any way to check if the MATLAB run is distributed among the cores by using the task manager? I believe this should make my code run faster, especially if it takes hours on a single core? Thanks, Ram

Answer 3

[John D'Errico]

MATLAB uses multiple cores only when there will be a gain. The problem is that multi-threading itself has overhead. So on simple, fast operations, there is no gain. It sometimes takes serious;y large problems before multiple cores kick in. Yes, I've gotten mine running at times, humming away, getting the fan to kick on, etc. But most of the time, on most operations, each specific call is done before you would even see multiple cores starting up. So if you are doing a lot of small operations on some problem, then no, multi-threading will never happen, and you simply won't gain from anything automatic.

If you are doing the same sets of operations on multiple problems, then yes, you can gain from using parallel processing, because you farm the whole operation out to separate processors, with one processor handling each separate problem. n processors, so it will approach an n-fold speedup. But again, there are costs in this, so you never gain a full n-fold speedup from using n cores.