Please enable JavaScript.
Coggle requires JavaScript to display documents.
Self Driving Car (Chapter 9 ADVANCED COMPUTER VISION) (Finding Lines I:…
Self Driving Car
(Chapter 9 ADVANCED COMPUTER VISION)
Steps we’ve covered so far:
Camera calibration (Ch 7)
Distortion correction (Ch 7)
Color/gradient threshold (Ch 8)
Perspective transform (Ch 7)
After doing these steps, you’ll be given two additional steps for the project:
Determine the lane curvature (Ch 9)
Detect lane lines (Ch 9)
Finding Lines I: Sliding Window
PEAKS IN HISTOGRAM
With this histogram we are adding up the pixel values along each column in the image. In our thresholded binary image, pixels are either 0 or 1, so the two most prominent peaks in this histogram will be good indicators of the x-position of the base of the lane lines. We can use that as a starting point for where to search for the lines. From that point, we can use a sliding window, placed around the line centers, to find and follow the lines up to the top of the frame. Then use sliding windows moving upward in the image (further along the road) to determine where the lane lines go.
SPLIT THE HISTOGRAM FOR TWO LINES
The first step we'll take is to split the histogram into two sides, one for each lane line.
SET UP WINDOWS AND WINDOW HYPERPARAMS
Choose the
number of sliding windows
Set the width of the windows
+/- margin
Set
minimum number of pixels
found to recenter window
ITERATE THROUGH windows TO TRACK CURVATURE
Loop through each window in nwindows
Find the boundaries of our current window. This is based on a combination of the current window's starting point (leftx_current and rightx_current), as well as the margin you set in the hyperparameters.
Use cv2.rectangle to draw these window boundaries onto our visualization image out_img. This is required for the quiz, but you can skip this step in practice if you don't need to visualize where the windows are.
Now that we know the boundaries of our window, find out which activated pixels from nonzeroy and nonzerox above actually fall into the window.
Append these to our lists left_lane_inds and right_lane_inds.
If the number of pixels you found in Step 4 are greater than your hyperparameter minpix, re-center our window (i.e. leftx_current or rightx_current) based on the mean position of these pixels.
FIT A POLYNOMIAL
Now that we have found all our pixels belonging to each line through the sliding window method, it's time to fit a polynomial to the line. First, we have a couple small steps to ready our pixels.
Concatenate the arrays of indices (previously was a list of lists of pixels)
Extract left and right line pixel positions of non zero pixels
Fit a second order polynomial to each using
np.polyfit
Finding Lines II: Search from Prior
You've now built an algorithm that uses sliding windows to track the lane lines out into the distance. However, using the full algorithm from before and starting fresh on every frame may seem inefficient, as the lines don't necessarily move a lot from frame to frame.
In the next frame of video you don't need to do a blind search again, but instead you can just search in a margin around the previous line position, like in the above image. The green shaded area shows where we searched for the lines this time. So, once you know where the lines are in one frame of video, you can do a highly targeted search for them in the next frame.
This is equivalent to using a customized region of interest for each frame of video, and should help you track the lanes through sharp curves and tricky conditions. If you lose track of the lines, go back to your sliding windows search or other method to rediscover them.
USE THE PREVIOUS POLYNOMIAL TO SKIP THE SLIDING WINDOW
In the previous quiz, we used left_lane_inds and right_lane_inds to hold the pixel values contained within the boundaries of a given sliding window. This time, we'll take the polynomial functions we fit before (left_fit and right_fit), along with a hyperparameter margin, to determine which activated pixels fall into the green shaded areas from the above image. Note that this margin can be a different value than the one originally used for your sliding windows!
To implement this in the below quiz, you'll want to grab only those pixels with x-values that are +/- your margin from your polynomial lines. Note that you'll only need to implement left_lane_inds and right_lane_inds in the quiz - most of the surrounding code, ignoring iterating through the windows, is the same as before!
The way we'll visualize this is a bit different than last time around, however, so make sure to pay attention to that if you want to visualize this step while working on your project.
Fitting on Large Curves
One thing to consider in our current implementation of sliding window search is what happens when we arrive at the left or right edge of an image, such as when there is a large curve on the road ahead. If minpix is not achieved (i.e. the curve ran off the image), the starting position of our next window doesn't change, so it is just positioned directly above the previous window. This will repeat for however many windows are left in nwindows, stacking the sliding windows vertically against the side of the image, and likely leading to an imperfect polynomial fit.
Can you think of a way to solve this issue? If you want to tackle the curves on the harder challenge video as part of the project, you might want to include this in your lane finding algorithm.
Another Sliding Window Search Approach
Another way to approach the sliding window method is to apply a convolution, which will maximize the number of "hot" pixels in each window. A convolution is the summation of the product of two separate signals, in our case the window template and the vertical slice of the pixel image.
You slide your window template across the image from left to right and any overlapping values are summed together, creating the convolved signal. The peak of the convolved signal is where there was the highest overlap of pixels and the most likely position for the lane marker.
Now let's try using convolutions to find the best window center positions in a thresholded road image. The code below allows you to experiment with using convolutions for a sliding window search function. Go ahead and give it a try.
