PCV Lab 2

Geodesy and Geoinformatics - Leibniz University

Projective Geometry provides a uniform framework for representing geometrical primitives such as points, lines, and planes. In addition, it can be used to represent the geometry of single images or image pairs. The main advantage of using this representation is that it leads to linear estimation problems, which can be solved without initial values.

The goal of this exercise is to obtain an advanced understanding of the use of projective geometry for determining the relative orientation of an image pair from hands-on experience. The Matlab files you need are provided in StudlP. Groups of two students should work together in this exercise.

The Fundamental Matrix and the Essential Matrix

Based on a set of corresponding points within two images, it is possible to determine the relative alignment of those two images with respect to each other. In the context of this exercise, two different approaches are studied for this purpose: The Fundamental Matrix, which can be determined without any further knowledge and the Essential Matrix, which can be determined if the calibration matrices are known for both images. As input, two sets of points are provided which were detected within two images showing the same scene. The correspondence of a point from the first list can be found in the second list by using the same index. Furthermore, the calibration matrix K is provided which applies for both images. 1) To avoid numerical issues, it is necessary to condition the image coordinates within the point sets before estimating the Fundamental or Essential matrix. For this purpose, the function ConditionTrafo has to be implemented. The function receives a set of points as argument and returns a 3 x 3 transformation matrix that can be used for conditioning (shift and scaling). This function should be used within the subsequent tasks.

2) Determine the Fundamental Matrix of the image pair. For this purpose, the eight-point algorithm has to be implemented in the function EightPointAlgo. The function receives two sets of points in the format described above. The return value of the function is a 3x3 matrix representing the Fundamental Matrix. Keep in mind to condition the points in the beginning and to recondition the matrix in the end.

3) Implement the function GetE8Point to determine the Essential Matrix E. For this purpose, use the calibration matrix K to normalise the image coordinates and the method FightPointAlgo (which was implemented in the previous task) to determine E. Like before, the function receives two sets of points in the format described above. The return value of the function is a 3x3 matrix representing the Essential Matrix.

Note that both functions (EightPointAlgo and GetE8Point) should be able to operate on >= 8 point pairs. Since outliers have to be expected within the point sets in general, it is appropriate to increase the robustness of the approach implemented so far. One option is to use random sample consensus (RANSAC). Therefore, in the subsequent tasks a RANSAC-based approach has to be implemented to determine the Essential Matrix. 4) 5) First, it is necessary to determine a suitable number of iterations for the RANSAC algorithm. It can be assumed that a set of known point pairs is specified from which eight are randomly chosen per iteration and which contains 30% outliers. What is the minimum number of iterations required, so that you can expect to select at least one subset without outliers with a probability of 99.5%? To assess the quality of an Essential Matrix derived by a single iteration of RANSAC, it is necessary to determine the size of the corresponding Consensus Set. For this purpose, implement the function DetermineConsensusSetScore. Determine the Fundamental Matrix F from the Essential Matrix of the current iteration and the known calibration matrix K. Use the fundamental matrix F to check the constraint x" - F - x' = O for every point pair. This can be done by computing the epipolar line |" = F- x' and comparing the Euclidean distance d of x" from |" to a threshold (e.g. d < 3 pixel). If the distance is smaller than the threshold, the point pair can be considered as an inlier; otherwise, it is an outlier. Note that d = I'"- x" / | all:

6) Implement the RANSAC-based approach in the function EstimateE. For this purpose, use the number of iterations obtained in subtask c) and the functions getE8Point and DetermineConsensusSetScore from the previous subtasks. In every iteration, a set of eight different point pairs is randomly chosen and used to estimate an Essential Matrix. After the last iteration, all point pairs from the set of inliers of the Essential Matrix with the highest consensus set score are used to estimate the final Essential Matrix.

7) In the last step, the relative orientation of the image pair is determined. For this purpose, the translation and rotation of the second image relative to the first one is estimated. This has to be done within the function DecomposeE by decomposing the matrix returned by EstimateE. This leads to two possible projection centres and two possible rotation matrices. To decide for one of the four solutions, the provided auxiliary function GetRe/Or can be used.