Render 1D column of Optimizati​onVariable​/Optimizat​ionExpress​ion objects to string?

In Matlab 2019a, I have a table wherein a column `IFpvfd` is an `OptimizationVariable` array:

myTable = table( [1:3]' , ...                                        
                 optimvar( 'IFpvfd' , 3 , 1 , 'Type','integer' , ... 
                           'LowerBound',0 , 'UpperBound',1 ) , ...   
                 'VariableNames' , {'RowNbr' 'IFpvfd'} )             
                                                                     
   myTable = 3×2 table                                               
       RowNbr                   IFpvfd                               
       ______ ___________________________________________            
         1    [1x1 optim.problemdef.OptimizationVariable]            
         2    [1x1 optim.problemdef.OptimizationVariable]            
         3    [1x1 optim.problemdef.OptimizationVariable]            
                                                                     
showvar( myTable.IFpvfd )                                            
                                                                     
   [ IFpvfd(1) ]                                                     
   [ IFpvfd(2) ]                                                     
   [ IFpvfd(3) ]                                                     

I want to add a column `IFpvfd2` that shows the `OptimizationVariable` more readably, either as a string array or cell array of character arrays:

RowNbr                   IFpvfd                    IFpvfd2   
______ ___________________________________________ _________ 
  1    [1x1 optim.problemdef.OptimizationVariable] IFpvfd(1) 
  2    [1x1 optim.problemdef.OptimizationVariable] IFpvfd(2) 
  3    [1x1 optim.problemdef.OptimizationVariable] IFpvfd(3) 

I was hoping that there was a function to do this using arrayfun:

myTable.IFpvfd2 = arrayfun( @SomeFunc , myTable.IFpvfd )

For `SomeFunc`, I don't think I can use an anonymous function based on `sprintf` because the formatting string only accepts primitive data:

@(x) sprintf( 'Some Formatting String' , x )

I can't use `showvar` or `writevar`, as they send the output to either console and a text file without providing a return argument.

Answer 1

[Walter Roberson]

myTable = table( [1:3]' , ...                                        
                 optimvar( 'IFpvfd' , 3 , 1 , 'Type','integer' , ... 
                           'LowerBound',0 , 'UpperBound',1 ) , ...   
                 'VariableNames' , {'RowNbr' 'IFpvfd'} )             

myTable = 3×2 table
    RowNbr                      IFpvfd                   
    ______    ___________________________________________

      1       [1×1 optim.problemdef.OptimizationVariable]
      2       [1×1 optim.problemdef.OptimizationVariable]
      3       [1×1 optim.problemdef.OptimizationVariable]

myTable.IFpvfd2 = rowfun( @SomeFunc , myTable, 'InputVariables', 'IFpvfd' )

myTable = 3×3 table
    RowNbr                      IFpvfd                        IFpvfd2 
                                                               Var1   
    ______    ___________________________________________    _________

      1       [1×1 optim.problemdef.OptimizationVariable]    IFpvfd(1)
      2       [1×1 optim.problemdef.OptimizationVariable]    IFpvfd(2)
      3       [1×1 optim.problemdef.OptimizationVariable]    IFpvfd(3)

function S = SomeFunc(OV)
  warnstate = warning('off', 'MATLAB:structOnObject');
  OVS = struct(OV);
  warning(warnstate);
  S = categorical(sprintf("%s(%d)", OVS.Name, OVS.OptimExprImpl.getVarIdx));
end

Note: it did not work to create SomeFunc as an anonymous function: for reasons I do not understand, MATLAB claimed there was no function with that name that was applicable to that datatype.

[FM]

Unfortunately, that doesn't seem to work for me. Here is my test function:

 1  % ScalrOptmVarName.m                                                                
 2  %-------------------                                                                
 3  % Return name of a scalar OptimVar                                                  
 4  % (i.e., optim.problemdef.OptimizationVariable) object                              
 5  function SOVname = ScalrOptmVarName(OV)                                             
 6     warnstate = warning('off', 'MATLAB:structOnObject');                             
 7     OVS = struct(OV);                                                                
 8     warning(warnstate);                                                              
 9     SOVname = categorical(sprintf("%s(%d)", OVS.Name, OVS.OptimExprImpl.getVarIdx)); 
10  end                                                                                 

I ran it on a table Tdecsn with field IFpvfd containing OptimVar (i.e., optim.problemdef.OptimizationVariable) objects:

Tdecsn.IFpvfd_cat = arrayfun( @ScalrOptmVarName , Tdecsn.IFpvfd )

The error is that the objects do not contain a Name field:

Reference to non-existent field 'Name'.                                              
Error in ScalrOptmVarName (line 9)                                                   
   SOVname = categorical(sprintf("%s(%d)", OVS.Name, OVS.OptimExprImpl.getVarIdx));

I drilled down into the OVS structure:

K>> OVS                                                                             
OVS = struct with fields:                                                           
                                                                                    
                       IndexNames: {{}  {}}                                         
                        Variables: [1×1 struct]                                     
                  IndexNamesStore: {{}  {}}                                         
                    OptimExprImpl: [1×1 optim.internal.problemdef.ExpressionForest] 
                         ExprType: Linear                                           
                             Size: [1 1]                                            
    OptimizationExpressionVersion: 2                                                
                                                                                    
K>> OVS.Variables                                                                   
ans = struct with fields:                                                           
    IFpvfd: [1642×1 optim.problemdef.OptimizationVariable]                          
                                                                                    
K>> OVS.OptimExprImpl                                                               
ans = ExpressionForest with no properties.                                          
                                                                                    
K>> OVS.Variables.IFpvfd                                                            
ans = 1642×1 OptimizationVariable array with properties:                            
                                                                                    
  Array-wide properties:                                                            
          Name: 'IFpvfd'                                                            
          Type: 'integer'                                                           
    IndexNames: {{}  {}}                                                            
  Elementwise properties:                                                           
    LowerBound: [1642×1 double]                                                     
    UpperBound: [1642×1 double]                                                     
                                                                                    
  See variables with showvar.                                                       
  See bounds with showbounds.                                                       
                                                                                    
                                                                                    
K>> x=fieldnames(OVS.Variables)                                                     
ans = 1×1 cell array                                                                
    {'IFpvfd'}                                                                      
                                                                                    
K>> x{:}                                                                            
ans = 'IFpvfd'                                                                      

I can use x{:} to access the name of the OptimVar object, but not its index in the array. It's index in the array doesn't necessarily match the Tdecsn row in which the object sits.

[Walter Roberson]

Try this...

 % ScalrOptmVarName.m                                                                
 %-------------------                                                                
 % Return name of a scalar OptimVar                                                  
 % (i.e., optim.problemdef.OptimizationVariable) object                              
 function SOVname = ScalrOptmVarName(OV)
     warnstate = warning('off', 'MATLAB:structOnObject');                             
     OVS = struct(OV);                                                                
     warning(warnstate);
     if hasfield(OVS, 'Name')
         varname = OVS.Name;
     else
         varnames = fieldnames(OVS.Variables);
         varname = varnames{1};
     end
     SOVname = categorical(sprintf("%s(%d)", varname, OVS.OptimExprImpl.getVarIdx)); 
 end                                                                                 

[FM]

Thanks! It works! In fact, I was coming back to see if getVarIdx works despite the fact that OptimExprImpl appears empty, and it did. So I was pleasantly unsurprised that your revised function works.

Just one small correction and one question.

The correction, at least in my version 2019a, is that Matlab doesn't recognize "hasfield", but changing that to "isfield" makes everything work.

The question is...how did you get so much insight into the inner workings of OptimVar? The help pages don't contain that level of detail.

[Walter Roberson]

Ah, I should indeed have coded isfield() not hasfield(), which is something completely different.

The question is...how did you get so much insight into the inner workings of OptimVar?

I used struct2cell() on two different variables, and then I used cellfun(@isequal) between the two . That told me that the only difference between them was in the apparently-empty OptimExprImpl . So I ran methods() on that field and saw getVarIdx which turned out to work.

[FM]

That's quite some sleuthing. Some tricks I will have to keep in my back pocket in case I'm against the wall in the future.

Thanks again!

Answer 2

[FM]

Here is one solution for a 1D column of OptimizationVariable objects and another solution for a 1D column of OptimizationExpression objects. The latter requires a temporary variable and can't be done in one statement. However, it is able to handle both cases. If I write an source m-file for the conversion function, therefore, I would use the 2nd scheme.

Both schemes assume a 1D column of objects. If the table column consists of a 2D array (which it can!), more sophisticated coding is needed.

% Scheme 1: Test data table with column of OptimizationVariable objects
myTable = table( [1:3]' , ...
                 optimvar( 'IFpvfd' , 3 , 1 , 'Type','integer' , ...
                           'LowerBound',0 , 'UpperBound',1 ) , ...
                 'VariableNames' , {'RowNbr' 'IFpvfd'} );

% Scheme 1: Render OptimizationVariable column into strings
myTable.strIFpvfd = splitlines( string( regexprep( ...
   strtrim( evalc( 'showvar( myTable.IFpvfd )' ) ) , ...
   '[ \[\]]' , '' ) ) );

   myTable = 3×3 table
    RowNbr                   IFpvfd                     strIFpvfd 
    ______ ___________________________________________ ___________
      1    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(1)"
      2    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(2)"
      3    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(3)"

% Scheme 2: Test data table with column of OptimizationExpression objects
myTable.IFpvfd2 = 0 + myTable.IFpvfd;

% Scheme 2: Render OptimizationExpression column into strings
noisyIFpvfd2 = splitlines( string( regexprep( ...
   strtrim( evalc( 'showexpr( myTable.IFpvfd2 )' ) ), ...
   '[ \[\]]' , '' ) ) );
myTable.strIFpvfd2 = noisyIFpvfd2(3:4:end)

   myTable = 3×5 table
    RowNbr                   IFpvfd                     strIFpvfd                     IFpvfd2                    strIFpvfd2 
    ______ ___________________________________________ ___________ _____________________________________________ ___________
      1    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(1)" [1x1 optim.problemdef.OptimizationExpression] "IFpvfd(1)"
      2    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(2)" [1x1 optim.problemdef.OptimizationExpression] "IFpvfd(2)"
      3    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(3)" [1x1 optim.problemdef.OptimizationExpression] "IFpvfd(3)"

% Show that Scheme 2 also works with OptimizationVariable
noisyIFpvfd2 = splitlines( string( regexprep( ...
   strtrim( evalc( 'showexpr( myTable.IFpvfd )' ) ), ...
   '[ \[\]]' , '' ) ) );
noisyIFpvfd2(3:4:end)

   ans = 3×1 string array
       "IFpvfd(1)"
       "IFpvfd(2)"
       "IFpvfd(3)"

So the function is:

% strOptVarExpr.m                                                     
%----------------                                                     
% Convert 1D column of OptimizationVariable or OptimizationExpression 
% objects into string representation of the symbolic representation   
function strOut = strOptVarExpr( OptVarExpr )                         
                                                                      
strOut = splitlines( string( regexprep( ...                           
   strtrim( evalc( 'showexpr( OptVarExpr )' ) ), ...                  
   '[ \[\]]' , '' ) ) );                                              
strOut = strOut(3:4:end);                                             
                                                                      
end                                                                   

The result is:

myTable.strIFpvfd = strOptVarExpr( myTable.IFpvfd );                                                                         
myTable.strIFpvfd2 = strOptVarExpr( myTable.IFpvfd2 )                                                                        
                                                                                                                             
   myTable = 3×5 table                                                                                                       
    RowNbr                   IFpvfd                     strIFpvfd                     IFpvfd2                    strIFpvfd2  
    ______ ___________________________________________ ___________ _____________________________________________ ___________ 
      1    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(1)" [1x1 optim.problemdef.OptimizationExpression] "IFpvfd(1)" 
      2    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(2)" [1x1 optim.problemdef.OptimizationExpression] "IFpvfd(2)" 
      3    [1x1 optim.problemdef.OptimizationVariable] "IFpvfd(3)" [1x1 optim.problemdef.OptimizationExpression] "IFpvfd(3)"