MATLAB Answers

What, if anything, can be done to optimize performance for modern AMD CPU's?

195 views (last 30 days)
Wick
Wick on 20 Apr 2018
Commented: Mark Smith on 5 Jan 2020 at 22:55
I have a AMD Threadripper CPU. I've seen suggestions that AMD-based computers will benefit from changing the math library from Intel's MKL to the AMD Core Math Library (ACML) along with recommendations to change the BLAS.SPEC file. Any threads relating to this seem quite old and it appears that AMD might have changed the name of its math library.
Is it still possible to optimize the math library for AMD's CPUs? If so, what do I need to do?
Thanks!

  23 Comments

Wick
Wick on 17 Nov 2019
My gains on TR were very much inline with yours percentage-wise. However, an FFT-heavy code didn't see any speed up. I suspect that one was bound by memory access. However, the matrix multiplication speedup is worth the hack alone! I'm hoping to upgrade to a new TR this season. The memory access should be superior.
Put up an answer so I can accept it. This is worth immortalizing!
Ned Flanders
Ned Flanders on 10 Dec 2019
@wick
Could you test on your TR whether disabled SMT is faster? Matlab starts one thread per Core (not available thread) on Intel and AMD CPUs and my quick test suggests that SMT off is somewhat faster. (another 5-10%)
Wick
Wick on 12 Dec 2019
I've tested SMT off in TR before. It's generally a bit faster that way. However, for my code at least, I'm still faster to run two instances of MATLAB and load up all cores. It's certainly not double, but it's something like 1.5X depending on the amount of memory access.

Sign in to comment.

Accepted Answer

Ned Flanders
Ned Flanders on 18 Nov 2019
Edited: Ned Flanders on 2 Dec 2019
You should read it if you want to understand the background.
EDIT: Before you start I have a short request for you that you could help me with and serve your own interest. Matlab will not implement this. If you think that Matlab should offer a permanent solution that serves all users independent of whether they use Intel or AMD CPUs, please make a feature request at Matlab to implement a nummeric library (e.g. OpenBLAS) that does not discriminate against non Intel CPUs. Mathworks will not make this change without people advocating for it. Thanks!
tl;dr
Solution 1 (Windows): Create a .bat file using e.g. "Notepad" with the following lines to start Matlab in AVX2 Mode.
@echo off
set MKL_DEBUG_CPU_TYPE=5
matlab.exe
Save as .txt and rename to .bat. If you double-click that file, Matlab will start the MKL in AVX2 Mode. If you start it the normal way, it will remain as always.
You can also download the .bat file from my highdrive if you trust me (which typically you should not as I am a random guy from the internet). If you delete the startup batch file provided in the download or the one you created yourself, its gone and your computer will be as it has been before.
Solution 2 (Windows): If you are happy with the results (which you surely will be :-)), you should make the setting permanent by entering MKL_DEBUG_CPU_TYPE=5 into the System Environment Variables. This has several advantages, one of them being that it applies to all instances of Matlab and not just the one opened using the .bat file.
Doing this will make the change permanent and available to ALL Programs using the MKL on your system until you delete the entry again.
LINUX: (Thanks to foreignrobot)
Simply type in a terminal:
export MKL_DEBUG_CPU_TYPE=5
and then run matlab from the same terminal.
For benchmarking, you can use this script:
To make it permanent in Linux edit your shell's configuration scripts (~/.bashrc for bash, ~/.zshrc for zsh etc) adding the line export MKL_DEBUG_CPU_TYPE=5. That'll apply in any newly opened shell and to apply it in an already open one simply do . ~/.bashrc or whatever your config script name is. (Thanks to lowpolybutt)
P.S.: In case you are on an older AMD FX CPU, you could test whether MKL_DEBUG_CPU_TYPE=4 works for you. This should enable AVX, but I haven't tested this.

  14 Comments

Pat Quillen
Pat Quillen on 4 Dec 2019
@Wick, you have to restart MATLAB every time. There's (currently) no way to force unloading the BLAS.
Valid separators between libraries on the list are space, semicolon, or comma---that's platform independent.
Wick
Wick on 4 Dec 2019
@PatQuillen thanks. That would explain why BLAS_VERSION cannot be set to something within the "Program Files" directory - the default location for MKL. It's breaking the parse at the space in the folder name. I had to copy the mkl.dll to another folder to try setting it as the fall-over library.
Yair Altman
Yair Altman on 17 Dec 2019
@Wick - try setting the Program Files path within quotation marks, as in "C:\Program Files\bla\bla\blas.dll";"C:\Program Files\bla2\bla2\blast.dll"

Sign in to comment.

More Answers (2)

Wick
Wick on 4 Dec 2019
Edited: Wick on 4 Dec 2019
The Accepted Answer by @NedFlanders is the best solution so far. Use that one.
But in the comments of that answer, MathWorks staff member @PatQuillen shared some useful undocumented environment variables that allow anyone to hook into any external BLAS library. This has the potential to be very useful for getting other BLAS packages such as OpenBLAS or BLIS working within MATLAB. Those comments are already buried in that thread so I'm pulling them out as a stand-alone solution. I'll repeat the warning from those comments: "This comes with the same caveats that MathWorks has not qualified our products against alternate BLAS implementations, so as above, we can't confirm that using your own BLAS, such as OpenBLAS, will work correctly throughout our products."
We're going to be setting environment variables for the host operating system. This can be done in a temporary or permanent way outside of MATLAB for both Linux and Windows as can be seen in @NedFlander's reddit thread. However, I'm only going to use the setenv command to set these for only one instance of MATLAB.
MATLAB loads the BLAS library the first time it tries to execute a computation so you can make changes before that point. Once the BLAS library is loaded, there's no way to unload it. You'll have to restart.
The environment variable BLAS_VERBOSITY will set MATLAB to detail information to the Command Window about the loading process. A value of 1 is minimal information, 2 is more.
setenv('BLAS_VERBOSITY','1')
The environment variable BLAS_VERSION, if defined, will direct MATLAB to load the BLAS library from the specified file(s) rather than use its built-in default. The files should be the .so/.dll for the BLAS implementation you plan to use that follows the symbols defined by https://netlib.org/blas/.
setenv('BLAS_VERSION','c:\temp\mkl.dll')
Would load a copy of the MKL library copied into c:\temp. You can use a semicolon, comma, or space to join multiple file locations into a single definition of BLAS_VERSION. So if you're writing your own BLAS that doesn't support all the possible methods you could roll over to MKL as a failsafe, e.g.

  0 Comments

Sign in to comment.


Ken M.
Ken M. on 9 Nov 2019
You are saying don't use windows for matlab.
Since that is the easy way around, people stop complaining about it after the switch.
Many programs just work faster with linux so if speed is important then linux is an obvious choice also for matlab.

  4 Comments

Show 1 older comment
Ken M.
Ken M. on 9 Nov 2019
Some compare test from Hardware Unboxed
If you don't mind the change, I wouldn't stay with windows. Overal I think it is a win for everyone. Maybe the main reason is that linux can handle more cores easier.
I also use an astronomy program pixinsight and it also works much faster on linux than windows.
Wick
Wick on 10 Nov 2019
Changing to Linux is not necessarily a possibility in the work flows for many people. Since Windows users on desktops vastly outnumber Linux users, it would be nice if Mathworks could compile with good options for Ryzen/Threadripper/Epyc users.
However, if someone does have the ability to run both operating systems back-to-back, I'd be more than happy to see a comparison of the two addressing per-thread performance.
Mark Smith
Mark Smith on 5 Jan 2020 at 22:55
Many thanks to Ned Flanders and others that have commented here. I have had some correspondence with Mathworks and indicated to them that Matlab should have optimized binaries for AMD cpus. As a lark, I decided to use the bench command in Win 10 and Kubuntu in various configurations to demonstrate the efficacy of having optimized binaries for AMD cpus. I primarily use Win 10 and am a Linux novice. I was able to get Kubuntu 19.10 installed on a separate drive and ran a few bench commands. I have 5 screen shots showing how use of the environmental variable changes performance in both Win 10 and Kubuntu which I was prevented from uploading. I also downloaded the AMD AOCL precompiled Ubuntu binaries and AMD Math Library (LibM) Ubuntu binary and used those with the bench command.
My machine for these tests is a Threadripper 2950x running on Asrock X399 Pro Gaming motherboard with Gigabyte 1080 Ti. For all these tests, the bios was set to defaults with Precision Boost Overdrive not active (3.5 Ghz base clock). I normally run Win 10 with the 2950x at 4.4 Ghz with powerful custom cooling. All tests were run with 64 GB ram.
The simple summary is that Win 10 is not suited for Matlab with a score of ~55 on the bar chart (below the Surface Pro 3). This is improved to ~75 using the Win 10 environmental variable MKL_DEBUG_CPU_TYPE=5. If I perform bench overclocked to 4.4 Ghz with 32 GB ram, the score goes to ~85. Not much jump for possibly running this fine chip at too high a clock.
For Kubuntu with bios default settings, no overclocking and no setting of MKL_DEBUG_CPU_TYPE=5 prior to starting Matlab results in a score of ~65. Adding export MKL_DEBUG_CPU_TYPE=5 prior to starting Matlab results in a score of ~95, thats right ~95. Adding export commands to link the AMD Ubuntu binaries along with MKL_DEBUG_CPU_TYPE=5 results in a score of ~98. With these changes it is the second ranked machine below entry for Win 7 running on Xeon E5-1650 v3 at 3.5 Ghz.
My takeaway is that if Mathworks major market is tied to Win 10, better math binaries are needed. The simple setting of an environmental variable is not sufficient in Win 10. The great benefit obtained in Linux by just setting the environment variable needs to be explained. I would be interested in knowing what is different between Linux and Win 10 that allows the MKL_DEBUG_CPU_TYPE=5 environmental variable to have such a much better effect in Linux. At this point I infer that Microsoft is also in bed with Intel in hampering AMD performance or just ignorant of the effects of its software on math libraries.
I like Matlab, it is powerful and a major tool in the industry. It needs to be better optimized for different cpu configurations.

Sign in to comment.

Sign in to answer this question.