You can try one of the following possible workaround for above problem:
1) Get the calibration right by getting more calibration images, especially towards the edges of the Field of View of the cameras to capture the distortion and skew with more accuracy. Trying out different calibration settings should also help in this.
2) Getting more overlap between the cameras and reducing the baseline also helps reducing the disparity and results in more reliable results. The disparityBM and disparitySGM functions also provide some control over parameters to allow for more pixels to be used in the disparity map ('ContrastThreshold', 'UniquenessThreshold').