MATLAB provides multiple libraries that mex code links to. There can be both 32-bit and 64-bit versions of functions in the various libraries. And there are mex build options that can force things to compile and link differently depending on the target. Add to that the fact that these libraries are constantly changing from release to release. A library function that originally listed an argument as int may change this to mwSize in a later release, and then change this yet again to size_t in an even later release. Or ptrdiff_t, etc. I agree it can be hard to keep up with these interface changes and at the same time ensure backward or forward compatability. The only advice I can give is to use the interface definitions from doc of the version you are using, and that should ensure that any build you do in that version will work properly. You will probably not be able to ensure portability across all releases no matter what you do, but using the interface definitions from the latest doc might be the most robust.
As far as using signed integers for indexing goes, there are advantages depending on how they are used in your code. Doing arithmetic with unsigned integers can be very tricky to avoid coding pitfalls, and it gets especially difficult if you end up mixing signed and unsigned integers and various sized integers in an expression. E.g., subtracting a larger signed index from a smaller signed index can result in a negative value, which would behave as expected if comparing to 0 for instance. Whereas if the integers were unsigned you might get a wrap-around to a positive result and then the comparing to 0 wouldn't work as expected. The reason I say "might" is because if the unsigned integers are shorter than int they will be converted to int first and you would get a negative result. So whether the result is negative or a wrap-around positive depends on the size of the unsigned integers involved. This could result in bugs in what seems like innocent comparison or loop indexing code. Different code may be needed for unsigned integers. I think it was stuff like this that motivated the developers of Java to avoid unsigned integers altogether.
Unfortunately, if you do use the official interface definitions from the doc, at some point in your programming life you will probably be forced to have expressions that involve both signed (e.g., mwSignedIndex) and unsigned (e.g., size_t) integers or integers of potentially different sizes because you will be comparing them or passing them into other functions with different integer argument types. E.g., simply getting a variable size from mxGetM (size_t) and passing it into a BLAS routine (mwSignedIndex) creates this issue. Every time this happens in your code a red warning light should go off in your head and you need to look very carefully at the code you have written to make sure it will behave as expected. Is one a shorter size than the other? What will be the intermediate types that the operands are converted to as the expression gets evaluated (i.e., "usual unary and binary conversions")? Will negative signed integer values get converted to positive unsigned values in an assignment or implicit argument conversion? Or vice-versa? Is mwSize always an unsigned integer in all releases of MATLAB? Etc., etc.
I guess the bottom line is to use the doc interfaces exactly and then be very careful when different integer types are used in an expression or are implicitly converted. And be extremely careful when doing arithmetic with unsigned integers.