菜鳥看論文——Disparities Matching Score

08.B-Spline Modeling of Road Surfaces for Freespace Estimation

96.A space-sweep approach to true multi-image matching.

98.Detection of Small Obstacles at Long Range Using Multibaseline Stereo

07.

A global optimiazion 

algorithm for real-time on-board stereo obstacle detection systems

The goal of free space calculation is to find the disparity value d of the obstacles which bound the free space. This disparity value may be different for every image column. For the image row, where the foot point of an obstacle touches the ground, this disparity becomes the same as the disparity value of the road surface.

The goal is to find the boundary v(d) respectively disparity d for every image column u which describes the free space as consistently as possible.

The best boundary is given by a maximal matching score. The key idea is to sum up a matching score for the road surface from the image bottom to the boundary and to sum up the matching score for objects with disparity d from the boundary in the image on upwards.The total score for row u and disparity d therefore writes as:

It is based on the plane sweep idea,He applied an edge filter on the input images and provided a geometric reconstruction of the scene. For every disparity value d the pixel (u;v) in the left image is compared with the pixel (u+d;v) in the right image. This corresponds to shifting the right image over the left image,Clearly, only obstacles with the correct disparity value are in focus and the gray values in the right and left images coincide. All other regions of the image seem to be out of focus. Carrying this idea from gray values to edges or edge directions as done in [4] is straight forward. Let EL;R(u;v) be the edge direction in the left and right image respectively at image position (u;v). The image based disparity score can then be computed as

 

 

 vmin and vmax are the upper and lower bound of the region of interest in the images. Essentially Equation 5 and 6 count the number of matches on the road between the obstacle and the camera and the number of matches for the image column u on any potential obstacle with the disparity d. For matches on the road surface, warping the right image onto the left one under the road homography yields better matching results (see [12]). However, a wrong orientation of the road surface will lead to wrong scores in the disparity score table because d(v) and v(d) will be incorrect.

Calculating the maximal disparity score for every image column independently leads to noisy

and unsatisfactory results. This is mainly due to stereo occlusion and low texture in the images. A way to solve this problem is to combine the results of neighboring image columns to reduce the influence of outliers and to smooth the result. Deviations in the result between neighboring image

columns are penalized by decreasing the total matching score. Algorithms which introduce such smoothness in a global optimum manner need a disparity-column matching score table for all possible disparities for each image column. Therefore the disparities matching score table has dimensions image width by disparity range. It encodes for every image column u and disparity d the likelyhood that d is the disparity of the road-obstacle boundary v(d). Optimization using this table is usually done by dynamic programming. For further details on this optimization step we refer to [4] or [1].

The DSI has been used in DP-based scanline optimization methods where a pixel of the DSI represents a matching score between a pixel of a reference scanline and that of the target scanline.

The proposed method modifies the DSI so that a pixel of the DSI represents a matching score for a column of pixels of the reference image under the road environment constraint.

1) Calculation of DSI: We set the region of interest (ROI) on the left (reference) image whose upper boundary is the vanishing line of the road plane and divide the image in the ROI into vertical columns of pixels (see Fig.8). Under the road environment constraint, a single disparity parameter

determines the whole correspondence for a column of pixels in the reference image since the y-coordinate of the roadobstacle boundary can be calculated from (2).

 2) Matching Score: In order to calculate the DSI, we haveto define the matching score which evaluate the goodness of the match. Most of the conventional stereo methods use matching score or cost based on the intensity difference between corresponding pixels or regions, such as sum of

squared differences (SSD) and sum of absolute difference (SAD). However, intensity-based matching measures are not robust enough under practical conditions. SAD and SSD rely on the constant luminance assumption. Therefore, they are sensitive to differences in camera gain or bias. Normalized correlation can compensate bias and multiplicative variation but is sensitive to outliers. Besides, image sampling tends to cause large intensity differences in textured regions unless

image registration is done with sub-pixel accuracy[2].

Compared to image intensity, the direction of an edge is stable under various lighting conditions since it is invariant with respect to bias and multiplicative variation. Since calculating gradients has blurring effect, we do not need sub-pixel image registration for the matching score calculation.

We calculate match scores only at salient edge pixels using the following simple binary score that compares the gradient vectors of the corresponding pixels: if the angle between the two vectors is smaller than the predefined threshold then score is 1 and otherwise score is 0. Edges are detected with Canny edge detector though we do not use hysterisis thresholding. Since scores for the unmatched edges are 0, outliers are just ignored and do not significantly affect the result. 

 

 

  Since the road plane is not front-parallel to the image planes, the directions of the corresponding gradient vectors in the road regions do not match well[13]. Therefore, we prepare the affine transformed right image with (1) and use that image for comparison of the road region pixels. By using this matching score, the criterion for the optimization becomes quite simple: to find the best path which gives the highest number of matched edge pixels.

 

If disparity values are known for pixels in the image, the plane sweep approach can be replaced by direct disparity measurements. This speeds up the calculation of the disparity score table because no sweep step is necessary. Let (u;v) be an image position and du;v the corresponding disparity value. The height Y(v;d) and distance Z(d) are computed by stereo triangulation. We define a disparity based score as

 

 We combine both approaches in this paper by adding the single scores for the image based approach with edges and the disparity based approach with correlation stereo. This combines the robustness of the direct disparity measurements and the density of edge information. Figure 4 shows the result of free space computation using a combination of the described algorithm under the planar road assumption as

commonly used in free space computation. This assumption holds for the close-by environment. Then the road rises to above one meter within the next 70 meters. The image based free space computation fails because the assumed displacement of the road surface beyond 50m has an offset of several pixels. The disparity based approach fails because the height of the road surface beyond 50m is above any

appropriate height threshold. The key limitation is that the algorithm requires to know the ground plane height changes,which are not modeled explicitly.