Organize randomly distributed grid points into a 2D matrix

Hello, I have a few pictures for calibrating images with a poster containing grid points. The picture is thresholded and points are extracted through regionprops. I then use the bounding box of the regions to recalculate the centroid with the actual grey levels of the image (more accurate in my opinion).

My question is that I have now an array of X(:) and Y(:) positions that give the positions of the grid on the image.

Now I want to organize these values into I(:) and J(:) values where I and J contain the point coordinate on the grid on the X and Y axis. Something like the point in the middle (doesn't matter the exact middle) is (0,0), the the point on the right is (1,0) and the one up is (0,1).

The problem is the image distortion. I cannot make a simple sorting algorithm like for example [~,index]=sort(X+1000*Y). But the image is regular enough to know the points at left right up and down of any point.

Here is an example image:

You should look at the closer and smaller points (around 10000 points) not the 42 big ones.

I did an algorithm that detects the closest point on the right of any point and below any point. This way:

dx=x'*ones(size(x))-ones(size(x))'*x;
dy=y'*ones(size(y))-ones(size(y))'*y;
xdist=(dx/10).^2+dy.^2;
ydist=dx.^2+(dy/10).^2;
right=xdist;right(dx<=0)=Inf;
down=ydist;down(dy<=0)=Inf;
[minx,mx]=min(right);
mx(minx>5)=0;
[miny,my]=min(down);
my(miny>5)=0;

This will make a 10000x10000 matrix of all distances from any point to any point distorted in x and y directions. And then take their minima (in the positive values). This is quite brutal for me...

After that 'mx' points to the index of the closest rightwards point for any point. And 'my' does the same thing. Then I do an algorithm with for loops to iterate starting from the center (0,0) until I fill all the points into I and J. Kind of like bucket tool in Paint. Remaining unfilled are left NaN. Anyway this for loop in MATLAB takes a lot of time.

Isn't there any tool in MATLAB to rearrange 2D grid points such as sort? But sort2D?

Cheers,

Amid.