problem performing multiple, independent tasks simultaneously in parfor

Here's a parfor code snippet that's intended to search for 2 different file types and load a file simultaneously, but fails due to different output types:

% run 3 independent things simultaneously to save time
parfor n 1:3
switch n
    case 1
        output{n} = dir('pathname\**\*.txt'); % this takes a long time
    case 2
        output{n} = dir('pathname\**\*.csv'); % this takes a long time
    case 3
        output{n} = load('pathname\matlab.mat'); % this also takes a long time
end
end

Here's how I'd prefer to write this function, which doesn't crash and runs MUCH faster than each function separately, but variables a, b, and c are not saved.

% run 3 independent things simultaneously to save time
parfor n 1:3
switch n
    case 1
        a = dir('pathname\**\*.txt'); % this takes a long time
    case 2
        b = dir('pathname\**\*.csv'); % this takes a long time
    case 3
        c = load('pathname\matlab.mat'); % this also takes a long time
end
end

I'm not I/O limited, these processes run simultaneously much faster. Run time with the second snippet is the same as the single longest operation. How can I run either method of code snippet with the outputs saved?

[Walter Roberson]

parfor n = 1:3

output{1} and output{2} would be directory information, output{3} would be a struct containing one field for each variable loaded from the matlab.mat file.

[Bruno Luong]

Redargless it does't work (I don't know why just by reading your code)

Your parfor doesn't do any real computation, just reading, so it is mostly limited by your perepherical HW (reading speed and transfert data through the bus). Not sur parfor will not in contrary to what you would expect slow down even more, since the caching is worse.

[David L.]

Edited above for clarity of my problem. Yes, the parfor loop runs much faster. Each function takes ~5 minutes, sequentially it's 15 minutes. In the parfor, it takes 5 minutes total.

Answer 1

[Walter Roberson]

That code that assigns to output{n} will not fail due to different output types. All three of the outputs (for the code you posted) would be struct(), and besides you are outputting a cell array.

parfor n = 1:3
    switch n
        case 1
            output{n} = dir('*.txt'); % this takes a long time
        case 2
            output{n} = dir('*.csv'); % this takes a long time
        case 3
            output{n} = struct('this', 5, 'that', 7); % this also takes a long time
    end
end

Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).

output

output = 1×3 cell array
    {0×1 struct}    {0×1 struct}    {1×1 struct}

Have you considered using parfeval?

F(1) = parfeval(@() dir('*.txt'), 1);
F(2) = parfeval(@() dir('*.csv'), 1);
F(3) = parfeval(@() struct('this', 5, 'that', 7), 1);
wait(F)
a = fetchOutputs(F(1))

a = 

  0×1 empty struct array with fields:

    name
    folder
    date
    bytes
    isdir
    datenum

b = fetchOutputs(F(2))

b = 

  0×1 empty struct array with fields:

    name
    folder
    date
    bytes
    isdir
    datenum

c = fetchOutputs(F(3))

c = struct with fields:
    this: 5
    that: 7

[David L.]

Thanks very much Walter! This solution worked great! I also tried a batch method that worked too.