addmulti

Add multiple key-value pairs to KeyValueStore

collapse all in page

Syntax

addmulti(KVStore,keys,values)

Description

addmulti(KVStore,keys,values) adds multiple key-value pairs to KVStore, which is a KeyValueStore created during mapreduce execution. Use addmulti in a map or reduce function written for use with mapreduce to store intermediate or final key-value pair information.

example

Examples

collapse all

Open Live Script

Use add and addmulti in map and reduce functions to pass data into the intermediate and final KeyValueStore. This example uses identity map and reduce functions that pass the inputs straight through to the output. The map and reduce functions are listed at the end of the example as local functions.

inds = tabularTextDatastore('airlinesmall.csv','SelectedVariableNames',...
    {'ArrDelay','DepDelay'},'TreatAsMissing','NA');
preview(inds)
ans=8×2 table
    ArrDelay    DepDelay
    ________    ________

        8          12   
        8           1   
       21          20   
       13          12   
        4          -1   
       59          63   
        3          -2   
       11          -1   


outds = mapreduce(inds,@myMapper,@myReducer,mapreducer(0));
********************************
*      MAPREDUCE PROGRESS      *
********************************
Map   0% Reduce   0%
Map  16% Reduce   0%
Map  32% Reduce   0%
Map  48% Reduce   0%
Map  65% Reduce   0%
Map  81% Reduce   0%
Map  97% Reduce   0%
Map 100% Reduce   0%
Map 100% Reduce  50%
Map 100% Reduce 100%

readall(outds)
ans=2×2 table
        Key               Value      
    ____________    _________________

    {'ArrDelay'}    {123523×1 double}
    {'DepDelay'}    {123523×1 double}

Local Functions

function myMapper(data,info,intermKV)
    addmulti(intermKV,{'ArrDelay' 'DepDelay'},{data.ArrDelay data.DepDelay});
end

function myReducer(key,intermValIter,outKV)
    data = getnext(intermValIter);
    while hasnext(intermValIter)
        data = [data; getnext(intermValIter)];
    end
    add(outKV,key,data);
end

Input Arguments

collapse all

Key-value pair storage object, specified as a KeyValueStore object. The mapreduce function automatically creates the KeyValueStore object during execution:

  • In the map function, the name of the intermediate KeyValueStore object is the third input argument to the map function, myMapper(data, info, intermKVStore). Use that same variable name to add intermediate key-value pairs with add or addmulti in the map function.

  • In the reduce function, the name of the final KeyValueStore object is the third input argument to the reduce function, myReducer(intermKey, intermValIter, outKVStore). Use that same variable name to add final key-value pairs with add or addmulti in the reduce function.

For more information, see KeyValueStore.

Keys, specified as a numeric scalar, numeric vector, character vector, string array, cell vector of character vectors, or cell vector of numeric scalars. If the keys are a numeric vector, cell vector, or string array, then each entry specifies a different key.

All of the keys added by the map function must have the same class. The keys added by the reduce function must also have the same class, but that class can differ from the class of the keys added by the map function.

Numeric keys cannot be NaN, complex, logical, or sparse.

Example: addmulti(intermKVStore,{'Sum'; 'Count'; 'Variance'},{sum(X); numel(X); var(X)}) adds three key-value pairs to an intermediate KeyValueStore object (named intermKVStore) using a cell vector to specify the keys.

Example: addmulti(intermKVStore,[1 2 3 4],{sum(X); mean(X); max(X); min(X)}) adds four key-value pairs to an intermediate KeyValueStore object using a numeric vector to specify the keys.

Example: addmulti(outKVStore,'Stats',{[mean(X) max(X) min(X) var(X) std(X)]}) adds a single key-value pair to a final KeyValueStore object (named outKVStore) using a character vector as the key.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | char | string

Values, specified as a cell array. Each entry in the cell array specifies the value in a key-value pair, so numel(values) must be equal to the number of keys. The entries in the cell array can be any MATLAB® object, including all valid MATLAB data types.

The OutputType argument of mapreduce affects the type of values that the reduce function can add:

  • If the OutputType is 'Binary' (the default), then a value added by the reduce function can be any MATLAB object.

  • If the OutputType is 'TabularText', then a value added by the reduce function can be a numeric scalar, character vector, or string scalar when using the add function. Additionally, you can use the addmulti function to add multiple values with a numeric vector, cell vector of character vectors, cell vector of numeric scalars, or string array. In each case, the numeric values cannot be NaN, complex, logical, or sparse.

Note

The above key-value pair requirements may differ when using other products with mapreduce. See the documentation for the appropriate product to get product-specific key-value pair requirements.

Example: addmulti(intermKVStore,{'Sum'; 'Count'; 'Variance'},{sum(X); numel(X); var(X)}) adds three key-value pairs to an intermediate KeyValueStore object named intermKVStore.

Example: addmulti(intermKVStore,[1 2 3 4],{sum(X); mean(X); max(X); min(X)}) adds four key-value pairs to an intermediate KeyValueStore object using a cell vector.

Example: addmulti(outKVStore,'Stats',{[mean(X) max(X) min(X) var(X) std(X)]}) adds a single key-value pair to a final KeyValueStore object named outKVStore.

Example: addmulti(outKVStore,{'Distance' 'Time'},{table.Distance table.Time}) adds two key-value pairs using variables in a table to specify the values.

Tips

  • Avoid using add in a loop, as it can negatively affect mapreduce execution time. Instead, use cell arrays to collect multiple values (using vectorized operations if possible) and use a single call to addmulti.

Version History

Introduced in R2014b

See Also

|

Topics