I suggest to look at the example MEX file and class wrapper in this GitHub repo. This is a solution I came up with a little while ago, and I just now saw this question. Hopefully this will be helpful to someone.
For your C++ class, class_type, mexFunction uses static data storage to hold a persistent (between calls to mexFunction) table of integer handles and smart pointers to dynamically allocated class instances. A std::map is used for this purpose, which facilitates locating known handles, for which only valid instances of your class are guaranteed to exist:
typedef unsigned int handle_type;
A std::shared_ptr takes care of deallocation when either (1) a table element is erased via the "delete" action or (2) the MEX-file is unloaded.
To prevent the MEX-file from unloading while a MATLAB class instances exist, mexLock is called each time a new C++ class instance is created, adding to the MEX-file's lock count. Each time a C++ instance is deleted mexUnlock is called, removing one lock from the lock count.
- Manage multiple persistent instances of a C++ class
- Small consecutive integer handles used in MATLAB (not cast pointers)
- Transparently handle resource management (i.e. MATLAB never responsible for memory allocated for C++ classes): (a) No memory leaked if MATLAB fails to issue "delete" action. (b) Automatic deallocation if MEX-file prematurely unloaded.
- Guard against premature module unloading
- Validity of handles implicitly verified without checking a magic number
- No wrapper class or functions mimicking mexFunction, just an intuitive switch-case block in mexFunction.
Note that these goals should be acheved without regard to any MATLAB class, but which can also help address memory management issues. As such, the resulting MEX-file can safely be used directly (but not too elegantly).
- Enumerate the different actions (e.g. New, Delete, Insert, etc.) in the Actions enum. For each enumerated action, specify a string (e.g. "new", "delete", "insert", etc.) to be passed as the first argument to the MEX function in MATLAB.
- Customize the handling for each action in the switch statement in the body of mexFunction (e.g. call the relevant C++ class method).