MATLAB Answers

Dynamic variable names for full workspace operations

16 views (last 30 days)
D. Plotnick
D. Plotnick on 1 Feb 2017
Commented: per isakson on 27 Feb 2017
To start with, I understand dynamic variable names are bad. I am not really trying to use them. What I really want to do is apply a specific operation to all variables in the current workspace; this way I can generate a generic function to apply that operation.
Two examples: Example 1
Let's say that I have a code where I can use double or single precision depending on user choice. I want to cycle through all of the workspace variables looking for e.g. doubles that have numel>1000 and convert all of them to single. I can use who to get my workspace, and then a loop with isa and a boolean to find all the variables that match those criteria. What I want to do now is perform the operation varname = single(varname) to reassign those variables to the single-precision class while keeping the same name. Is there a way to do this other than using dynamic variable names?
Example 2
Lets say I ran into an "out-of-memory" error on the GPU because there is a bunch of junk left on there from other operations. I want to cycle through all gpuArray class variables and pull them down using varname = gather(varname), perform a reset(gpuDevice), and then possibly place them back on the gpu using varname = gpuArray(varname). Again, I understand that I could write a code that knows all of the variable names, the point here is to generate a generic code that can do the operation on all the correct workplace variables.
Again, if there is a totally obvious way of doing this that doesn't involve dynamic names, please let me know. Also, if there is something super bad about either of these concepts, I need to know that too.
Otherwise...how do you code something like this using dynamic variable names, since Matlab seems to make that kind of operation intentionally difficult.
Thanks for your help, -Dan
  4 Comments
D. Plotnick
D. Plotnick on 10 Feb 2017
I did not realize data sharing was handled in that way; that is extremely helpful to know. I had thought that kind of sharing only applied when the data was stored in a handle class.

Sign in to comment.

Accepted Answer

per isakson
per isakson on 3 Feb 2017
Edited: per isakson on 8 Feb 2017
There are good reasons to avoid eval (Here, I use eval as shorthand for eval, evalin and assignin), see
"Example 1" &nbsp I don't think there is a solution without eval. But after all, eval exists in several languages and that's for a reason - I assume.
Here is my attempt to answer "Example 1".
M1 = ones(2e4)+eps;
M2 = ones(1e4)+eps;
variables = reshape( whos('M*'), 1,[] );
for v = variables
convert( v.name, 'single' )
end
whos('M*')
prints
Name Size Bytes Class Attributes
M1 20000x20000 1600000000 single
M2 10000x10000 400000000 single
where
function convert( variable_name, new_class )
% convert variable, variable_name, to type, new_class, in the workspace of the caller
%
% assert that the value of variable_name is the name of a variable in the caller
xpr = sprintf( 'exist( ''%s'', ''var'' );', variable_name );
num = evalin( 'caller', xpr );
%
if num == 1
% str = sprintf( '%1$s = cast( %1$s, ''%2$s'' );', variable_name, new_class );
% sts = evalin( 'caller', str );
% Error: The expression to the left of the equals sign
% is not a valid target for an assignment.
xpr = sprintf( 'cast( %s, ''%s'' );', variable_name, new_class );
try
assignin( 'caller', variable_name, evalin( 'caller', xpr ) );
catch me
fprintf( 2, 'Error: ''%s''\n', me.message );
end
else
fprintf( 2, 'Undefined variable, ''%s''\n', variable_name );
end
end
Stephen Cobeldick presents the following list of problems related to eval. I argue that my above use of eval avoids most of these problems.
  • Slow &nbsp the conversion in the above code is as fast as &nbsp M1=cast(M1,'single'); M2=cast(M2,'single');
  • Buggy &nbsp No, not in this case. convert does one thing and it's possible to test it thoroughly.
  • Security Risk &nbsp Not in this case. All necessary tests may be done in convert.
  • Difficult to Work With &nbsp The use of convert should not cause any problems.
  • Obfuscated Code Intent &nbsp convert communicates the intent well enough.
  • Confuses Data with Code &nbsp Not applicable in this case.
  • Code Helper Tools do not Work &nbsp That's true in this case, but F1 works with convert.
&nbsp
ADDENDUM, 2017-02-08
An improved version of convert inspired by the comments by Jan Simon
function convert( variable_name, new_type )
% convert variable, variable_name, to type, new_type, in the workspace of the caller
narginchk( 2, 2 )
assert( isa( variable_name, 'char' ), 'convert:IllegalClass'...
, '"%s" is not a character array', value2short(variable_name) )
assert( isrow( variable_name ), 'convert:IllegalSize' ...
, '"%s" is not a row', value2short(variable_name) )
assert( isvarname( variable_name ), 'convert:IllegalName' ...
, '"%s" is not a valid variable name', variable_name )
assert( isa( new_type, 'char' ), 'convert:IllegalClass' ...
, 'The type of new_type, %s, is not a char', value2short(new_type) )
assert( isrow( new_type ), 'convert:IllegalSize' ...
, 'The value of new_type, %s, is not a row', value2short(new_type) )
type_list = {'int8','uint8','int16','uint16','int32','uint32' ...
,'int64','uint64','double','single','logical','char'};
assert( any(strcmp( new_type, type_list )), 'convert:IllegalType' ...
, 'The value of new_type, %s, is not a valid type name', new_type )
% assert that the value of variable_name is the name of a variable in the caller
xpr = sprintf( 'exist(''%s'', ''var'' );', variable_name );
assert( evalin('caller',xpr) == 1, 'convert:UndefinedVariable' ...
, '"%s" is not a defined variable', variable_name )
cmd = sprintf( 'builtin( ''cast'', %s, ''%s'' );', variable_name, new_type );
try
assignin( 'caller', variable_name, evalin( 'caller', cmd ) );
catch me
fprintf( 2, 'Error: "%s"\n', me.message );
end
end
where
function str = value2short( val )
% value2short converts value to a short string that is suitable to display
%
% See also: mat2str
%
if nargin > 0
str = workspacefunc( 'getshortvalue', val );
max_len = 48;
if length( str ) >= max_len
str = [ str(1:max_len-4 ), ' ...' ];
end
else
str = 'NIL';
end
end
  12 Comments
per isakson
per isakson on 27 Feb 2017
@Stephen Cobeldick, Thank you for your answer. I agree fully regarding "good design" and "no unknown variables".
I assumed as a premise that OP had painted himself into a corner. After reading the question more carefully I realize that OP posed the question out of curiosity.

Sign in to comment.

More Answers (2)

Edric Ellis
Edric Ellis on 2 Feb 2017
For the gpuArray case, you could simply use save and load, i.e.
tempFile = tempname();
save(tempFile);
reset(gpuDevice);
load(tempFile);
delete(tempFile);
  2 Comments
Walter Roberson
Walter Roberson on 3 Feb 2017
I did some poking around and thought I was getting somewhere but it didn't work. I was looking for a way to get at the workspace of the current function, with the idea that altering the workspace would be equivalent to altering the variable. I found that if you declare a nested function and use functions() that you get a workspace of the nested function that includes all variables in the parent assigned at the point you took the handle, which seemed like a doable way of getting access to your own workspace. Unfortunately changing the workspace did not change the variables in the function even for the shared variables. I was not able to get further on this.
It did leave me wondering if it would work for moving values in and out of the GPU array. If you have a shared variable that is assigned a gpu array and you gather it and send it again, then does that affect the original gpu array? The gather is going bring it back clearly, but the rewrite might instead create a second variable. I consider evalin('caller') to be a form of eval() though others might disagree I guess.

Sign in to comment.


Joss Knight
Joss Knight on 5 Feb 2017
Edited: Joss Knight on 5 Feb 2017
Well, if you're really serious about a tool for managing storage of GPU arrays, then you need a new class. This would be a numeric handle type that forwards all its functions to the underlying type, and adds all new objects to a static list. All functions run in a try...catch statement to catch parallel:gpu:array:OOM and, if triggered it calls a static utility function to gather the contents of the list back to the host and try again.
The only difficulty here is that you need to provide an implementation of every single method you want your new type to implement, i.e. every method of gpuArray (and a few more that aren't methods of gpuArray but are functions that can take gpuArray inputs). But that code could be autogenerated fairly easily.
  2 Comments
Joss Knight
Joss Knight on 17 Feb 2017
It's just a boiler-plate method for any function, so, say, for plus:
function varargout = plus(varargin)
% This bit swaps out the custom-type arguments for
% their underlying gpuArray property
for i = 1:numel(varargin)
if (isa(varargin{i}, 'MyManagedGPUArrayType')
varargin{i} = varargin{i}.UnderlyingArrayProperty;
end
end
% Try at least twice
for i = 1:2
try
[varargout{1:nargout}] = plus(varargin{:});
catch me
if i == 2 || me.identifier ~= "parallel:gpu:array:OOM"
rethrow(me);
else
MyManagedGPUArrayType.doSomeGatheringToClawBackMemory();
continue;
end
end
break;
end
end
So you create some script that reads a long list of function and creates a file with all these forwarding methods in, substituting in the name of the function. Well, no, you'd create a utility function for most of this call-gather-call structure and have a much simpler repeated boiler-plate for each method.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by