Apparently, the function rgb2hsv can return three output arguments. This line of code explicitly ignores the last two. A similar result can be obtained by implicitly ignoring them, by just retrieving one output.
hueChannel = rgb2hsv(videoFrame)
The use of the ~ is more commonly used to retrieve later output arguments and ignoring the earlier ones. As an example:
A = [1 4 5 3] ;
[maxValue, maxIndex] = max(A) % retrieve both
[~, maxIndex] = max(A) % use this if you are only interested in the second output
You have to look in the help of rgb2hsv to see what the first output variable will hold exactly.
