MATLAB system command bug - returns partial stdout

Hi,

I am seeing some very strange and perplexing behaviour with the system function. OS is Ubuntu 12.04 64-bit, MATLAB version is 2010a. Here's some example code:

[status, result] = system('find /home -name "*txt"');
disp(result);

Assuming no file system changes I expect the same results every time, however this is not what I'm seeing. Occasionally stdout results are partially returned and the remainder is piped to stdout AFTER the system function finished, so is seen as output to the terminal. The only hint as to what's going on is that this appears to happen more on machines with slow I/O - this is, find might be taking longer to pull out the results.

My suspicions, in order of what I think to be the most likely cause,

1. Threading or select bug implemented by the system call when receiving and combining stderr and stdout from the called process - impossible to dig further here obviously.
2. stdout buffering
3. Related to stdin not being directed from /dev/null as per the MATLAB doc "tips" somehow corrupting the output ... though I doubt this. The output is delivered, just not when/where it should be.

If anyone has any idea what might be causing this I'd love to hear it.

Cheers Jess

Answer 1

[LP]

Hi Jess,

We haven't worked out how to fix this either. We made a "brute" force wrapper but be sure to use only passive system calls (eg ls, find, etc)

function [status, result] = brute_system(command)
 % [status, result] = brute_system(command)
 %
 % MATLAB BUG causes imporoer buffering from system command so here we
 % attmept to fix the bug by
 %
 % Removing stdin buffering with                stdbuf -o0 
 % Ensuring no random input from stdin with     < /dev/null
 %
 % And in the event this doesn;t work, we rerun the command twice untill we
 % get the correct reponse

   newCommand = ['stdbuf -o0 ',command,' </dev/null'];

   dblConfirmation = false;
   cnt = 1;

   while(~dblConfirmation)
       [status1, result1] = system(newCommand);
       [status2, result2] = system(newCommand);

       if strcmp(result1,result2) && status1 == status2
           dblConfirmation = true;
           status = status1;
           result = result1;
       else
           cnt = cnt+1;
           fprintf('\n%-20s :: Command %s failed, trying again - Attempt #%d',mfilename, newCommand, cnt);
       end

   end

end

[Andreas Goser]

Hello LP, when you write "we", I assume multiple members on your team have this issue. Did you contact Technical Support and your workaround is part of their suggestion?

Answer 2

[LP]

Sorry - we is the collective me but yes, fairly robust occurrence of this. Can quite easily simulate it too. Run this code and just type into the command window (eg hit 's' at any time!).

Further this has been documented on stack overflow but that solution did not work here (tried - note how i modify the input command in 'brute_system' code above)

http://stackoverflow.com/questions/23948227/safer-alternative-to-matlabs-system-command

for i=1:20000

      tic;
[status, txt]= system(sprintf('ls -md %s/*',someDir));            

      fprintf('\nGo %3d took %5.2fs,',i, toc);    
      if i==1
          last_txt = txt;
      end

      if any(~strcmpi(txt, last_txt))
          error('oops')        
      end    

end

Answer 3

[Jeremy Hughes]

Very late to the story here, but there was a bug in the linux kernal that affected the system command in this way. https://www.mathworks.com/support/bugreports/details/1400063