Error while running linkage function

I want to create dendrogram from a large distance matrix (2583x2583). So, I'm using linkage function to create a tree of hierarchical clusters. My distance matrix is not symmetric. So, I'll be doing this with lower and upper diagonal. Since I already have the distance matrix in a square form, I needed to get it in the 'pdist' output format. For that, I used 'squareform' function. Following is the code I am using.

 load correl; %A correlation coefficient matrix
 correlDist = 1 - correl;
 lhd = tril(correlDist,-1) + (tril(correlDist,-1))'; %get lower half-diagonal
 %Get the distance matrix in the pdist output format
 lhdPdist = squareform(lhd);
 lZ = linkage(lhdPdist,'average');

I am getting the following error:

 ------------------------------------------------------------------------
           Access violation detected at Tue Sep 26 16:41:49 2017
 ------------------------------------------------------------------------

 Configuration:
   Crash Decoding     : Disabled
   Default Encoding   : windows-1252
   Graphics card 1    : Intel Corporation ( 0x8086 ) Intel(R) HD Graphics Version 
 0.0.0.0
   MATLAB Architecture: win64
   MATLAB Root        : C:\Program Files\MATLAB\R2014b
   MATLAB Version     : 8.4.0.150421 (R2014b)
   Operating System   : Microsoft Windows 7 Professional 
   Processor ID       : x86 Family 6 Model 58 Stepping 9, GenuineIntel
   Software OpenGL    : 1
   Virtual Machine    : Java 1.7.0_11-b21 with Oracle Corporation Java HotSpot(TM) 
 64-Bit Server VM mixed mode
   Window System      : Version 6.1 (Build 7601: Service Pack 1)

 Fault Count: 1

 Abnormal termination:
 Access violation

 Register State (from fault):
   RAX = 000000000032e229  RBX = 0000000023cf0060
   RCX = 000000000032e446  RDX = 0000000000000222
   RSP = 0000000004025040  RBP = 0000000000000227
   RSI = 00000000000009c6  RDI = 000000000032d501

    R8 = 0000000025659a18   R9 = 0000000000000052
   R10 = 0000000000000a16  R11 = 00000000000009c5
   R12 = 000000000000012e  R13 = 0000000000000002
   R14 = 0000000000000130  R15 = 000000000000004f

   RIP = 000000000ed93c7a  EFL = 00010213

    CS = 0033   FS = 0053   GS = 002b

 Stack Trace (from fault):
 [  0] 0x000000000ed93c7a C:\Program 
 Files\MATLAB\R2014b\toolbox\stats\stats\private\linkagemex.mexw64+00015482
 [  1] 0x000000000ed9a4c8 C:\Program 
 Files\MATLAB\R2014b\toolbox\stats\stats\private\linkagemex.mexw64+00042184
 [  2] 0x000000000eda0baf C:\Program 
 Files\MATLAB\R2014b\toolbox\stats\stats\private\linkagemex.mexw64+00068527 
 mexFunction+00000143
 [  3] 0x00000000fc5f3701 C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmex.dll+00079617 mexRunMexFile+00000129
 [  4] 0x00000000fc5f2762 C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmex.dll+00075618 inSwapMexfileReader+00000690
 [  5] 0x00000000fc5f2248 C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmex.dll+00074312 mexUnlock+00006008
 [  6] 0x00000000052605e3 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_dispatcher.dll+00067043 
 Mfh_file::dispatch_fh+00000659
 [  7] 0x0000000005260aee C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_dispatcher.dll+00068334 
 Mfunction_handle::dispatch+00000766
 [  8] 0x000000000534b606 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00112134 
 MathWorks::MException::IMExceptionData::IMExceptionData+00037094
 [  9] 0x000000000534aa20 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00109088 
 MathWorks::MException::IMExceptionData::IMExceptionData+00034048
 [ 10] 0x0000000005349219 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00102937 
 MathWorks::MException::IMExceptionData::IMExceptionData+00027897
 [ 11] 0x0000000005348de5 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00101861 
 MathWorks::MException::IMExceptionData::IMExceptionData+00026821
 [ 12] 0x000000000539b708 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00440072 
 inPathNotification::function_delete_notification+00128952
 [ 13] 0x0000000005373ea5 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00278181 
 inGetNameInSymbolTable+00022053
 [ 14] 0x0000000005371b4f C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00269135 
 inGetNameInSymbolTable+00013007
 [ 15] 0x0000000005371ab1 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00268977 
 inGetNameInSymbolTable+00012849
 [ 16] 0x000000000539406f C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00409711 
 inPathNotification::function_delete_notification+00098591
 [ 17] 0x00000000053927af C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00403375 
 inPathNotification::function_delete_notification+00092255
 [ 18] 0x0000000005391359 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00398169 
 inPathNotification::function_delete_notification+00087049
 [ 19] 0x00000000052605e3 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_dispatcher.dll+00067043 
 Mfh_file::dispatch_fh+00000659
 [ 20] 0x0000000005260aee C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_dispatcher.dll+00068334 
 Mfunction_handle::dispatch+00000766
 [ 21] 0x000000000534b606 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00112134 
 MathWorks::MException::IMExceptionData::IMExceptionData+00037094
 [ 22] 0x000000000534aa20 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00109088 
 MathWorks::MException::IMExceptionData::IMExceptionData+00034048
 [ 23] 0x0000000005349219 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00102937 
 MathWorks::MException::IMExceptionData::IMExceptionData+00027897
 [ 24] 0x0000000005348de5 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00101861 
 MathWorks::MException::IMExceptionData::IMExceptionData+00026821
 [ 25] 0x000000000539b708 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00440072 
 inPathNotification::function_delete_notification+00128952
 [ 26] 0x0000000005393fa8 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00409512 
 inPathNotification::function_delete_notification+00098392
 [ 27] 0x00000000053927af C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00403375 
 inPathNotification::function_delete_notification+00092255
 [ 28] 0x0000000005391359 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00398169 
 inPathNotification::function_delete_notification+00087049
 [ 29] 0x00000000052607a4 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_dispatcher.dll+00067492 
 Mfh_file::dispatch_fh+00001108
 [ 30] 0x0000000005260aee C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_dispatcher.dll+00068334 
 Mfunction_handle::dispatch+00000766
 [ 31] 0x00000000053c4a88 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00608904 
 inPathNotification::function_delete_notification+00297784
 [ 32] 0x00000000053c4a04 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00608772 
 inPathNotification::function_delete_notification+00297652
 [ 33] 0x00000000053c49c5 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00608709 
 inPathNotification::function_delete_notification+00297589
 [ 34] 0x00000000053c4992 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00608658 
 inPathNotification::function_delete_notification+00297538
 [ 35] 0x00000000053c4947 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00608583 
 inPathNotification::function_delete_notification+00297463
 [ 36] 0x000000000533d69d C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+00054941 inEvalExp+00001309
 [ 37] 0x000000000546525d C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+01266269 
 inEvalCmdWithLocalReturn+00000285
 [ 38] 0x0000000005465181 C:\Program 
 Files\MATLAB\R2014b\bin\win64\m_interpreter.dll+01266049 
 inEvalCmdWithLocalReturn+00000065
 [ 39] 0x000000000688e31d C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmwbridge.dll+00058141 mnGetPrompt+00001517
 [ 40] 0x000000000688ed09 C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmwbridge.dll+00060681 mnParser+00000745
 [ 41] 0x00000000fd0dd834   C:\Program 
 Files\MATLAB\R2014b\bin\win64\mcr.dll+00383028 
 mcrInstance::mnParser_on_interpreter_thread+00000036
 [ 42] 0x00000000fd0a68e7   C:\Program 
 Files\MATLAB\R2014b\bin\win64\mcr.dll+00157927 
 mcr::runtime::setInterpreterThreadToCurrent+00019751
 [ 43] 0x00000000fd0a6923   C:\Program 
 Files\MATLAB\R2014b\bin\win64\mcr.dll+00157987 
 mcr::runtime::setInterpreterThreadToCurrent+00019811
 [ 44] 0x00000000fd0a7121   C:\Program 
 Files\MATLAB\R2014b\bin\win64\mcr.dll+00160033 
 mcr::runtime::setInterpreterThreadToCurrent+00021857
 [ 45] 0x00000000fab5d3a6   C:\Program 
 Files\MATLAB\R2014b\bin\win64\uiw.dll+00512934 UIW_AttachThreadInput+00001270
 [ 46] 0x00000000fab5cc35   C:\Program 
 Files\MATLAB\R2014b\bin\win64\uiw.dll+00511029 wsd_to_MSW+00004373
 [ 47] 0x00000000fab5ccb9   C:\Program 
 Files\MATLAB\R2014b\bin\win64\uiw.dll+00511161 wsd_to_MSW+00004505
 [ 48] 0x00000000773887b2                     
 C:\Windows\system32\USER32.dll+00165810 GetMenuBarInfo+00000626
 [ 49] 0x000000007736f587                     
 C:\Windows\system32\USER32.dll+00062855 SystemParametersInfoW+00000247
 [ 50] 0x0000000077374815                     
 C:\Windows\system32\USER32.dll+00083989 IsProcessDPIAware+00000453
 [ 51] 0x00000000775cbc65                      
 C:\Windows\SYSTEM32\ntdll.dll+00310373 KiUserCallbackDispatcher+00000031
 [ 52] 0x000000007737908a                     
 C:\Windows\system32\USER32.dll+00102538 PeekMessageW+00000182
 [ 53] 0x000000007737506e                     
 C:\Windows\system32\USER32.dll+00086126 GetKeyState+00000238
 [ 54] 0x0000000077373a07                     
 C:\Windows\system32\USER32.dll+00080391 PeekMessageA+00000087
 [ 55] 0x000007fef1750a83 C:\Program Files\Bitdefender Antivirus 
 Free\atcuf\262824135760000000\atcuf64.dll+00133763
 [ 56] 0x000007fef173479a C:\Program Files\Bitdefender Antivirus 
 Free\atcuf\262824135760000000\atcuf64.dll+00018330
 [ 57] 0x00000000000200c6                                   <unknown-
 module>+00000000
 [ 58] 0x0000000000000001                                   <unknown-
 module>+00000000
 [ 59] 0x000036556a35ef26                                   <unknown-
 module>+00000000
 [ 60] 0x000000000402f430                                   <unknown-
 module>+00000000
 [ 61] 0x00000000ffffffff C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmwservices.dll+00065535 
 svSetDisplayQueryFunctions+00018159
 [ 62] 0x000000000402f4a9                                   <unknown-
 module>+00000000
 [ 63] 0x00000000ffffffff C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmwservices.dll+00065535 
 svSetDisplayQueryFunctions+00018159
 [ 64] 0x00000000ffffffff C:\Program 
 Files\MATLAB\R2014b\bin\win64\libmwservices.dll+00065535 
 svSetDisplayQueryFunctions+00018159
 [ 65] 0x000000000e03fbc0                                   <unknown-
 module>+00000000

 This error was detected while a MEX-file was running. If the MEX-file
 is not an official MathWorks function, please examine its source code
 for errors. Please consult the External Interfaces Guide for information
 on debugging MEX-files.

 If this problem is reproducible, please submit a Service Request via:
     http://www.mathworks.com/support/contact_us/

 A technical support engineer might contact you with further information.

 Thank you for your help.** This crash report has been saved to disk as 
 C:\Users\DEEPA\AppData\Local\Temp\matlab_crash_dump.2328-1 **

 Caught MathWorks::System::FatalException
 [Please exit and restart MATLAB]>>

When I run lZ = linkage(lhdPdist); then no errors are given. Only when I give 'average' option as method I get the above erro. I went through some posts here. It seems like MATLAB is expecting another input. But while I am using distance matrix, I don't think I need any other option. Please suggest.

Answer 1

[Vladimir Calderón]

I tried making a copy custom of linkage script to ignore using the linkagemex, which is where the bug is, and use the linkageold function within linkage.m script, i,e commenting th if sentence and always use the else sentence of these lines:

if exist('linkagemex','file')==3
    % call mex file
    if passX
        if (memEff)
            %call the memory efficient algorithm
            Z = linkagemex(Y',method,{'euc'},memEff); %note that Y is transposed here
        else
           Z = linkagemex(Y,method,pdistArg, memEff);
        end
    else
        Z = linkagemex(Y,method);
    end
    
    if any(strcmp(method,{'av' 'wa' 'co' 'we'}))
    % if 'ave','ward', 'com' or 'weighted average' is used, we need to 
    % re-arrange the rows in Z matrix by sorting the third rows of Z matrix.
      Z = rearrange(Z);
    end
else
    warning(message('stats:linkage:NoMexFilePresent'));
    if passX
        Y = pdist(Y,pdistArg{:});
    end
    % optional old linkage function (use if mex file is not present)
    Z = linkageold(Y,method);
end

However it became horribly slow, so I'm just ignoring some kinds of linkage.

The best solution would be debugging linkagemex.

Answer 2

[Yussif M. Awelisah]

Please Prof. Arnab Roy, I have tried using the Windowed raise cosine empirical mode decomposition but i hvave been struggling a bit with matab code for the algorithm.

Please I will be grateful if you can help me with the matlab code.

Email: [e-mail]