Compute the projective rectification of weakly calibrated binocular stereo images.

A binocular stereo setup is called weakly calibrated if the fundamental matrix, which describes the projective relation between the two images, is known. Rectification is the process of finding a suitable set of transformations, that transform both images such that all corresponding epipolar lines become collinear and parallel to the horizontal axes. The rectified images can be thought of as aquired by a stereo configuration where the left and right image plane are identical and the difference between both image centres is a horizontal translation. Note that rectification can only be performed if both of the epipoles are located outside the images.

Typically, the fundamental matrix is calculated beforehand with match_fundamental_matrix_ransac and FMatrix is the basis for the computation of the two homographies H1 and H2, which describe the rectifications for the left image and the right image respectively. Since a projective rectification is an underdetermined problem, additional constraints are defined: the algorithm chooses the set of homographies that minimizes the projective distortion induced by the homographies in both images. For the computation of this cost function the dimensions of the images must be provided in Width1, Height1, Width2, Height2. After rectification the fundamental matrix is always of the canonical form

                  /  0  0  0 \
                  |  0  0 -1 |  .
                  \  0 +1  0 /

In the case of a known covariance matrix CovFMat of the fundamental matrix FMatrix, the covariance matrix CovFMatRect of the above rectified fundamental matrix is calculated. This can help for an improved stereo matching process because the covariance matrix defines in terms of probabilities the image domain where to find a corresponding match.

Similar to the operator gen_binocular_rectification_map the output images Map1 and Map2 describe the transformation, also called mapping, of the original images to the rectified ones. The parameter Mapping specifies whether bilinear interpolation ('bilinear_map') should be applied between the pixels in the input image or whether the gray value of the nearest neighboring pixel should be taken ('nn_map'). The size and resolution of the maps and of the transformed images can be adjusted by the parameter SubSampling, which applies a sub-sampling factor to the original images. For example, a factor of two will halve the image sizes. If just the two homographies are required Mapping can be set to 'no_map' and no maps will be returned. For speed reasons, this option should be used if for a specific stereo configuration the images must be rectified only once. If the stereo setup is fixed, the maps should be generated only once and both images should be rectified with map_image; this will result in the smallest computational cost for on-line rectification.

When using the maps, the transformed images are of the same size as their maps. Each pixel in the map contains the description of how the new pixel at this position is generated. The images Map1 and Map2 are single channel images if Mapping is set to 'nn_map' and five channel images if it is set to 'bilinear_map'. In the first channel, which is of type int4, the pixels contain the linear coordinates of their reference pixels in the original image. With Mapping equal to 'no_map' this reference pixel is the nearest neighbor to the back-transformed pixel coordinates of the map. In the case of bilinear interpolation the reference pixel is the next upper left pixel relative to the back-transformed coordinates. The following scheme shows the ordering of the pixels in the original image next to the back-transformed pixel coordinates, where the reference pixel takes the number 2.

                +---+---+
                | 2 | 3 |
                +---+---+
                | 4 | 5 |
                +---+---+

Based on the rectified images, the disparity be computed using binocular_disparity. In contrast to stereo with fully calibrated cameras, using the operator gen_binocular_rectification_map and its successors, metric depth information can not be derived for weakly calibrated cameras. The disparity map gives just a qualitative depth ordering of the scene.

gen_binocular_proj_rectification is reentrant and processed without parallelization.

match_fundamental_matrix_ransac, vector_to_fundamental_matrix

map_image, projective_trans_image, binocular_disparity

gen_binocular_rectification_map

J. Gluckmann and S.K. Nayar: ``Rectifying transformations that minimize resampling effects''; IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2001, vol I, pages 111-117.

3D Metrology