Scanning reference image and its distorted images

I am writing code where it scanned the folder of reference images and distorted images. I want to match the reference image with its distorted images so that I can assess the quality later. I have about 29 reference images and 145 distorted images. The distorted images are in random order when I downloaded it. I tried match it by using size unfortunately other reference images also have the same size thus the result I get are wrong. Is there any unique features that reference images share with their distorted images? This is my code.

ref_info = dir('Image database\imgref\*.bmp');            %search directory of reference images
dis_info = dir('Image database\imgdist\*.bmp');           %search directory of distorted images    

for z = 1 : 145
    dist_name = strcat('Image database\imgdist\',dis_info(z).name);
    imgDist = imread(dist_name);        
    for y = 1 : 29 
    ref_name = strcat('Image database\imgref\',ref_info(y).name);
    imgRef = imread(ref_name);
        if (size (imgRef) == size (imgDist))
          similarity = ImageAssessment(imgRef,imgDist);
          Quality_Score(z,:) = similarity;
        else
        end
    end
end