Compute the distance values for a rectified stereo image pair.

binocular_distance computes pixel-wise correspondences between two images of a rectified stereo rig using correlation techniques. Different from binocular_distance this operator transforms these pixel correlations into distances of the corresponding 3D world points to the stereo camera system.

The algorithm requires a reference image Image1 and a search image Image2 which must be rectified, i.e., corresponding epipolar lines are parallel and lie on identical image rows ( r1=r2 ). In case this assumption is violated the images can be rectified by using the operators binocular_calibration, gen_binocular_rectification_map and map_image. Hence, given a pixel in the reference image Image1 the homologous pixel in Image2 is selected by searching along the corresponding row in Image2 and matching a local neighborhood within a rectangular window of size MaskWidth and MaskHeight. For each defined reference pixel the pixel correspondences are transformed into distances of the world points defined by the intersection of the lines of sight of a corresponding pixel pair to the z=0 plane of the rectified stereo system. These distances are returned in the single channel image Distance. For this transformation the rectified internal camera parameters CamParamRect1 of camera 1 and CamParamRect2 of camera 2, and the the external parameters RelPoseRect have to be defined. Latter characterizes the relative pose of both cameras to each other and specifies a point transformation from the rectified camera system 2 to the rectified camera system 1. These parameters can be obtained from the operator binocular_calibration and gen_binocular_rectification_map. After all, a quality measure for each distance value is returned in Score, containing the best result of the matching function S of a reference pixel. For the matching, the gray values of the original unprocessed images are used.

The used matching function is defined by the parameter Method allocating three different kinds of correlation:

'sad': Summed Absolute Differences  
       S(r1,c1,d) := SUM(|g1(r1',c1')-g2(r1',c1'+d)|), 0=<=S<=255,
'ssd': Summed Squared Differences   
       S(r1,c1,d) := SUM((g1(r1',c1')-g2(r1',c1'+d))²), 0<=S<=65025, 
'ncc': Normalized Cross Correlation 
       S(r1,c1,d) :=
         SUM(g1(r1',c1')-gm1(r1,c1))*(g2(r1',c1'+d)-gm2(r1,c1+d)))/
         SQRT(SUM((g1(r1',c1')-gm1(r1,c1))²)*
              SUM((g2(r1',c1+d)-gm2(r1,c1+d))²)),
       -1.0<=S<=1.0,

whereas SUM(x) is the averaged summation of x within a window of
size (2m+1) x (2n+1). Especially gm(r,c) is the mean value of
g(r',c') within this window at row r and column c. g1 and g2 are the gray
values of the unprocessed input image.

The size of the correlation window has to be odd numbered and is passed in MaskWidth and MaskHeight. The search space is confined by the minimum and maximum disparity value MinDisparity and MaxDisparity. Due to pixel values not defined beyond the image border the resulting domain of Distance and Score is generally not set along the image border within a margin of height MaskHeight/2 at the top and bottom border and of width MaskWidth/2 at the left and right border. For the same reason, the maximum disparity range is reduced at the left and right image border.

Since matching turns out to be highly unreliable when dealing with poorly textured areas, the minimum variance within the correlation window can be defined in TextureThresh. This threshold is applied on both input images Image1 and Image2. In addition, ScoreThresh guarantees the matching quality and defines the maximum ('sad','ssd') or, respectively, minimum ('ncc') score value of the correlation function. Setting Filter to 'left_right_check', moreover, increases the robustness of the returned matches, as the result relies on a concurrent direct and reverse match, whereas 'none' switches it off.

The number of pyramid levels used to improve the time response of binocular_distance is determined by NumLevels. Following a coarse-to-fine scheme disparity images of higher levels are computed and segmentated into rectangular subimages to reduce the disparity range on the next lower pyramid level. TextureThresh and ScoreThresh are applied on every level and the returned domain of the Distance and Score images arises from the intersection of the resulting domains of every single level. Generally, pyramid structures are the more advantageous the more the distance image can be segmented into regions of homogeneous distance values and the bigger the disparity range must be specified. As a drawback, coarse pyramid levels might loose important texture information which can result in deficient distance values.

Finally, the value 'interpolation' for parameter SubDistance increases the refinement and accuracy of the distance values. It is switched off by setting the parameter to 'none'.

// ...
// read the internal and external stereo parameters
read_cam_par ('cam_left.dat', CamParam1)
read_cam_par ('cam_right.dat', CamParam2)
read_pose ('relpose.dat', RelPose)

// compute the mapping for epipolar images
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, RelPose, 
                                 'geometric', 'bilinear', CamParamRect1,
                                 CamParamRect2, Cam1PoseRect1, Cam2PoseRect2,
                                 RelPoseRect)

// compute the distance values on online images 
while 1
  grab_image_async (Image1, FGHandle1, -1)
  map_image (Image1, Map1, ImageMapped1)

  grab_image_async (Image2, FGHandle2, -1)
  map_image (Image2, Map2, ImageMapped2)

  binocular_distance(ImageMapped1, ImageMapped2, Distance, Score, 'sad',
                     11, 11, 20, -40, 20, 2, 25, 'left_right_check',
                     'interpolation')
endwhile

binocular_disparity returns 2 (H_MSG_TRUE) if all parameter values are correct. If necessary, an exception handling is raised.

binocular_distance is reentrant and automatically parallelized (on domain level).

map_image

threshold

binocular_disparity

map_image, gen_binocular_rectification_map, binocular_calibration, distance_to_disparity, disparity_to_distance

3D Metrology