Mex API Wish List

18 vues (au cours des 30 derniers jours)
James Tursa
James Tursa le 14 Juin 2013
Commenté : James Tursa le 17 Fév 2021
I thought I would start a thread to list all of the desired changes, upgrades, and additions to the official mex API. So here goes my first cut at a list:
.
----------------------------------------
DOCUMENT EXISTING BEHAVIOR
----------------------------------------
.
1) Fully document the garbage collection behavior for mxArray variables and memory, and how it is affected by the various API functions. For example (but not limited to):
- The fact that using mxSetCell, mxSetField, mxSetFieldByNumber removes the input mxArray from the garbage collection list, and that they change portions of the input mxArray (e.g., change the variable type to 3 indicating sub-element). Also that they work only once on the same input (to put the same mxArray in another element you must bump up the Reference Count ... but the official API includes no functions for doing this).
- The fact that mxSetPr, mxSetPi, mxSetIr, mxSetJc remove the input address from the garbage collection list.
- None of the addresses in the mxArray header (pr, pi, ir, jc) are on the garbage collection list ... their disposition is determined entirely by what happens to the variable they are a part of.
.
2) Fully document variable sharing types and how you get it. E.g.,
- Shared Data Copy (e.g., what you commonly get when you do Y = X)
- Shared Reference Copy (e.g., same mxArray address listed multiple times in a cell array, reference count > 0)
- Shared Parent Copy (e.g., the cell array is shared with another cell array, so elements of one cell array are effectively shared with elements of another cell array, even though there is no CrossLink or Reference Count indicator of this at the element level)
- Shared Handle Copy (e.g., classdef object derived from the handle class)
.
3) Document what happens when you exit a mex routine. E.g.,
- Shared data copies of the plhs[ ] variables are returned, then everything on the garbage collection list is destroyed.
.
4) Fully document what happens when you call mxDestroyArray. E.g., as near as I can tell this is what actually happens:
If ( Reference Count > 0 ) Then
Decrement Reference Count by 1
Else
If ( This is not a shared data copy ) Deep Free all the data areas
Free the mxArray header
Remove mxArray header from the garbage collection list if it is there
Remove mxArray header from the CrossLink and Reverse Crosslink lists if it is a shared data copy of something
Endif
.
--------------------------------------------------------------------
MAKE OFFICIAL SOME UNDOCUMENTED API FUNCTIONS
--------------------------------------------------------------------
.
It would be nice if some of the more useful currently undocumented API functions could be officially documented. Creating shared data copies and reference copies is an essential capability for efficiently working with large variables in mex routines. For example (not a complete list):
.
5) mxCreateSharedDataCopy (returns shared data copy)
6) mxGetReference (returns reference count)
7) mxCreateReference (returns input with reference count bumped up by 1)
8) mxUnshareArray (unshares array suitable for cell array manipulation)
9) mxUnreference (decrements reference count by 1)
10) mxIsSharedArray (returns sharing status)
11) mxSetReferenceCount (sets reference count)
12) mxCreateUninitDoubleMatrix (returns double matrix with uninitialized data ... fast)
13) mxCreateUninitDoubleArray (returns double array with uninitialized data ... fast)
14) mxCreateUninitNumericArray (returns numeric array with uninitialized data ... fast)
15) mxFastZeros (returns double matrix with 0's ... fast)
16) mxGetUserBits (gets highest 8 bits of variable flags)
17) mxSetUserBits (sets highest 8 bits of variable flags)
18) mxIsA (checks to see if variable is a specified class)
.
---------------------------
ADD NEW FUNCTIONS
---------------------------
.
19) Complementary functions for putting mxArray variables and memory back on the garbage collection lists once they have been removed. E.g., something like:
- mexMakeArrayTemporary (reverse of mexMakeArrayPersistent)
- mexMakeMemoryTemporary (reverse of mexMakeMemoryPersistent)
- Note: Latter already has an undocumented function available, mxAddToAllocList
.
20) Add inquiry functions for determining when an address is on the garbage collection list. E.g., something like:
- mexIsArrayPersistent
- mexIsMemoryPersistent
- Note: Latter already has an undocumented function available, mxIsOnAllocList
.
21) Add function to get the actual pointer of a property, not a deep copy of it. E.g.,
- mxGetPropertyPtr
.
22) Add functions to set a property directly, or a shared data copy, without a deep copy. E.g.,
- mxSetPropertyPtr
- mxSetPropertySharedDataCopy
.
23) Add functions to determine the nearby mxArray variables in the shared data copy linked list. E.g.,
- mxGetNextSharedDataCopy (returns next mxArray * in CrossLink linked list)
- mxGetPrevSharedDataCopy (returns previous mxArray * in Reverse CrossLink linked list)
.
24) Add function for determining parent (e.g., cell or struct or class). It is recognized that reference copies may have multiple parents. E.g.,
- mxGetParents (returns list of all immediate parents)
.
25) Add smarter versions of mxSetCell, mxSetField, mxSetFieldByNumber that take into account the variable type and current sharing status of the variable and automatically update the reference count etc in a smart way. E.g., something like this:
void mxSetCellSmart( mxArray *mx, mwIndex i, mxArray *cell )
{
int VariableType;
mxArray *cell_copy;
if ( mx == NULL ) return;
if ( i < 0 ) return;
VariableType = mxGetVariableType( cell );
if ( VariableType == 4 ) { // if temporary
mxSetCell( mx, i, cell );
} else if ( VariableType == 3 ) { // if already a sub-element of cell or struct
mxSetCell( mx, i, cell );
mxCreateReference( cell );
} else {
cell_copy = mxCreateSharedDataCopy( cell );
mxSetCell( mx, i, cell_copy );
}
}
.
26) Add function & macro for determing MATLAB version. E.g.,
- matlab_version( ) would return MATLAB version currently running
- MATLAB_VERSION would be a macro indicating what MATLAB version is being used to compile (e.g., 0x2012b)
.
27) Add thread-safe functionality to the API, or create separate thread-safe functions for creating mxArray variables, etc.
.
28) All of the above for Fortran. And fix several of the Fortran example programs (some have bugs and will not work as expected for certain inputs, or will not work at all in a 64-bit environment)
  6 commentaires
ANDREY
ANDREY le 2 Nov 2013
I fully support item 28) All of the above for Fortran. Although Eigen lib in C++ provides a very convinient way to write mex files it would be even more natural to write them in Fortran 90/2003 if Mathworks makes a really nice API (based on the existing one by James) and Intel give the users Array Visualizer back on 64 bit systems.
James Tursa
James Tursa le 17 Fév 2021
Update: The (still undocumented) API functions mxGetPropertyShared and mxSetPropertyShared were added to the C library in R2018a.

Connectez-vous pour commenter.

Réponses (2)

Jan
Jan le 19 Juil 2013
Modifié(e) : Jan le 19 Juil 2013
Documentation for these functions would support the work with the API also:
mxFevalFunctionHandle, mexRunMexFile, mexSetCallbacks
mxTranspose, mxTransposeArray, mxInPlaceBlockTranspose
mxTxtCmp, mxCreateStringFromNChars, mxGetNChars,
mxIsExactSharedCopy, mxIsIdentical
mxArgMustBeIndices, mxColonop_fill_with_doubles, mxCreateConvertedCopy
mxGetFields, mxMallocEx, mxSerialize
utAccurateEstimateCPUSpeed, utEqualWithinEps
I do not know a method to access files from the MEX side using the File-ID from opening the file in Matlab. Tunneling the access back to the Matlab level by mexCallMATLAB is not efficient enuogh for my case. I thought of asking a new question shortly before I found this thread. But now let me append this as further enhancement request.

Pavel Holoborodko
Pavel Holoborodko le 20 Juil 2013
@James
Mangled C++ names are very useful in reversing undocumented functions - since it easy to derive original signature of the function: arguments and return type.
You can write small wrapper to use mangled C++ functions in C. Something like this would work:
mexext.h:
extern "C"
mxArray* mxGetPropertySharedC(const mxArray*, unsigned int, const char * );
mexext.cpp (compile with C++ compiler):
mxArray* mxGetPropertySharedC(const mxArray* pa,
unsigned int index,
const char *propname)
{
return mxGetPropertyShared(pa,index,propname);
}
And you are right of course - my note on mxDestroyArray is unnecessary. I have fixed my post and added some findings on mxArray_tag.
My previous comment got folded, so I repeat my link once more: Undocumented MEX API
  3 commentaires
Pavel Holoborodko
Pavel Holoborodko le 20 Juil 2013
Is there way to install C and C++ compilers separately these days? Usually both of them are distributed in the same package (if not as one executable). Or is it some special compiler like Watcom or similar?
James Tursa
James Tursa le 17 Fév 2021
Update: The (still undocumented) API functions mxGetPropertyShared and mxSetPropertyShared were added to the C library in R2018a.

Connectez-vous pour commenter.

Catégories

En savoir plus sur Write C Functions Callable from MATLAB (MEX Files) dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by