Lane Level Localization with Camera and Inertial Measurement Unit using an Extended Kalman Filter

Abstract

This thesis studies a technique for combining vision and inertial measurement unit (IMU) data to increase the reliability of lane departure warning systems. In this technique, 2nd-order polynomials are used to model the likelihood area of the location of the lane marking position in the image as well as the lane itself. An IMU is used to predict the drift of these polynomials and the estimated lane marking when the lane markings can not be detected in the image. Subsequent frames where the lane marking is present results in faster convergence of the model on the lane marking due to a reduced number of detected erroneous lines.

A technique to reduce the effect of untracked lane markings has been employed which bounds the previously detected 2nd-order polynomial with two other polynomials within which lies the likelihood region of the next frame's lane marking. Similarly, the slope at each point on the lane marking model lies within a certain range with respect to the previously detected 2nd-order polynomial. These bounds and slope employ similar characteristics as the original line; therefore, the lane marking should be detected within the bounded area and within the expected slope range given smooth transitions between each frame.

An inertial measurement unit can provide accelerations and rotation rates of a vehicle. Using an extended Kalman filter, information from the IMU can be blended with the last known coefficients of the estimated lane marking to approximate the lane marking coefficients until the lane is detected. A measurement of the position within the lane can be carried out by determining the number of pixels from the center of the image and the estimated lane marking. This measurement value can then be converted to its real-world equivalent and used to estimate the position of the vehicle within the lane. Also, the heading of the vehicle can be determined by examining the distance from the vanishing point of the camera and the vanishing point of the lane markings.