Roadway Debris Cognition for Self-Driving Cars


By Dr. Lance B. Eliot, the AI Insider for AI Trends and a regular contributor

You are driving along on the open highway and enjoying the scenery. There aren’t many other cars around and you have opted to put the pedal to the floor. With the top down on your convertible, you have the wind blowing through your hair and you feel like a million bucks. What could mar this perfect picture of driving on the open roads? Roadway debris. Suppose that you suddenly notice up ahead of you that there is an object laying on the road. Upon first spying the object, you can’t quite discern what it is. You can tell that it is in your lane. It seems to be several inches high and a few feet wide. Sitting in the middle of your lane, you are pretty sure that you will need to go around it, rather than trying to roll over it. Looking over your shoulder, you check to see if the lane next to you is available so that you can get out of your current lane.

Nearing the object, you can more clearly discern what it is. It’s a blown-out tire.  The rubber is torn and distorted, but it clearly is the bulky remnant of a tire. Looks like the type of tire that would be on a SUV, definitely larger than the tire on a regular passenger car, and smaller than a tire on a truck.  You steer your convertible into the next lane. Now, you are getting really close to the tire. You zip past it and can see more details of the ripped-up rubber. Your guess is that somehow the busted tire ended-up on the highway and probably other cars or trucks have bashed it time and again, rolling over it and pushing it back-and-forth. You glance in your rearview mirror, hoping that other cars upcoming will realize that the tire is sitting there in the middle of the roadway. At night time, it would be hard to see and likely cars and trucks will continue to run over it.  Hopefully no one will get panicked and get into an accident because of the debris.

As a human driver, you undoubtedly see roadway debris quite frequently. In my many years of driving the freeways in Los Angeles, I believe that I’ve seen most everything that has ever dropped onto the road. Clothing is a popular item, often consisting of shoes or coats. Not sure how people are losing their shoes and coats while otherwise driving on the freeway, but it seems to be a common occurrence. Lots of car parts are often seen. If I had stopped to pick-up every side-view mirror that I’ve seen on the freeway, I’d have a warehouse full of them.  Ladders sometimes are on the freeway, which makes a lot of sense to me. There are tons of contractors that hang ladders on the side or top of their pick-up trucks, and the odds that they have diligently secured the ladder to the vehicle is probably low. I’ve seen sofas, chairs, and all kinds of furniture on the freeway, usually spilled out from a moving van that was not well packed and secured.

Fortunately, the debris that I’ve seen has seldom caused an accident in my presence.  Listening constantly to the traffic reports on the radio, I know that these debris happenings do often cause accidents. There are other instances of debris getting onto the roadway, such as when a tanker truck overturns and spills something toxic or flammable onto the street.  For today’s column, I am going to focus on normal everyday debris. Things that are extraordinary such as dangerous fluids and such will be covered in a later column.

We have every right to assume and believe that a self-driving car should be able to handle roadway debris. Currently, the debris handling capabilities of AI self-driving cars is extremely simplistic and not able to contend with anything complicated. Let’s walk through the steps involved in dealing with roadway debris and we’ll cover how the AI should be coping with this common and potentially deadly malady.

The first step involves having the self-driving car detect that debris exists and should be given due consideration. Using the various sensors on the self-driving car, the AI should be continually ascertaining whether there is any debris up ahead. Similar to how a human does it, the AI will likely get an initial clue that there is debris on the road, but not be quite sure until getting nearer to the object. The further away the detection occurs is good, since it gives the system time to analyze the situation to figure out what to do. Plus, the earlier it is detected the greater usually the number of options about what to do about the debris.

The cameras on the self-driving car will typically provide visual images that can be scanned to try and detect debris on the roadway. Not a sure thing, though, since the cameras might not have a good angle on the item, or the cameras might be blocked by a car ahead or by dirt sitting on the lenses of the camera. Pairing the visual images with radar, the AI system can try to determine if the radar is pinging off the debris and also therefore affirming that there might be something on the road. LIDAR images are also examined.  Using all of the sensory capabilities, a sensor fusion occurs and the AI needs to try and pick the needle out of the haystack.  A false positive (falsely thinking that there is debris), can occur, in which case this is possibly Okay as long as the self-driving car does not take sudden and risky action based on a false belief. A false negative (falsely concluding that there isn’t debris even though there is debris), can be a serious issue since the self-driving car might end-up hitting the debris or otherwise get into serious driving troubles.

Besides trying to detect if the debris exists, there are other crucial parameters to be determined. What is the distance to the debris? Is the debris in the path of the self-driving car? Is the debris in motion or sitting motionless? How high up does the debris extend? How large is the debris? Does it occupy just a portion of the lane or extend across lanes? These and numerous other aspects need to be rapidly assessed. All of these factors play into what to do about the debris. For some of these questions there won’t be a complete answer at first. Only once the self-driving car gets closer to the object will these answers become more fully known.  Meanwhile, based on probabilities and conjecture, the AI needs to be assessing what to do, even if it does not yet have a perfect understanding of what the debris is and where it is.  Just like humans, assessing the situation involves uncertainties and estimations, utilizing imperfect awareness.  If the AI system waits until a perfect understanding has been reached, precious time might have evaporated that could have been used to make decisions to act and carry out needed actions.

By determining the distance to the object and knowing the speed of the self-driving car itself, the AI can begin to formulate plans of what to do. Usually, there will already be templates about various circumstances involving debris. In other words, rather than acting as though the AI has never experienced roadway debris, the AI developers will have hopefully gotten the AI system prepared for the debris circumstances. This can also be done via machine learning, whereby prior situations of debris in the roadway have been captured and analyzed, and the AI system makes use of those prior situations to gauge whether the current situation is applicable to prior experience.

Now the self-driving car needs to figure out what action to take.  Should the self-driving car slow down and possibly come to a stop in front of the debris? This might be prudent in some cases, while in other cases this might be disastrous. Imagine if there are other cars behind the self-driving car and they are anxious to continue speeding along the road. Having the self-driving car come to a halt in the middle of the freeway could trigger a domino effect of multiple car crashes. With human drivers this happens too, such as the time that I saw a car driven by an older man come to a halt to avoid hitting a stray bumper laying on the road. Other cars behind him were swerving like mad to avoid hitting him.  Their swerving caused the other cars to nearly get hit. It was a mess.

Maybe the self-driving car should try to straddle the debris, rolling over it. But this needs to be carefully considered. Suppose the debris hits the underbelly of the car and damages something? Another approach involves avoiding the debris by switching lanes. Sometimes human drivers swerve momentarily out of their lane into the next lane over. This can be dangerous if there are other cars coming along in those lanes. The self-driving car might opt to switch lanes entirely.

All of these options need to be considered and re-assessed continually. A choice made at a distance of a thousand feet away might not be available anymore if the self-driving car has advanced on the debris and is now only 100 feet away. Awareness of the debris should be increasingly available, which also then causes a re-calibration of the danger and options to select from. You might wonder why not just have the self-driving car automatically switch lanes the moment that it detects debris?  This is a rather simplistic solution. It could be that the other lanes aren’t open to move into. It could be that there is only one lane. It could be that the moving into another lane might not solve the issue if the debris extends across more than one lane. Etc.

Another aspect that needs to be considered is the other traffic involved in the circumstance. The self-driving car has to assume that other cars will potentially react to the debris. A human driver might suddenly swerve into the path of the self-driving car, or otherwise take an action that limits the options available to the self-driving car. In fact, another car might hit the debris, prior to the self-driving car reaching the debris, and shove it over into the path of the self-driving car wherein beforehand perhaps it wasn’t in that path. Or, another car might hit the debris and cause it to splinter, sending fragments all along the roadway.

This brings up that another option for the self-driving car involves hitting the debris. Sometimes, options are so limited that the “best” recourse is to just go ahead and strike the debris. This happened to me one day on a blind curve on a mountain. I came around the bend and there was debris sitting on the road. I could not come to a halt as there were other cars behind me that would have rammed into me. I did not see the debris until the last moment because it was sitting at the corner of the blind curve. I could not have swerved into the oncoming lane of traffic because I would have produced a head-on collision. I ran right into the debris, and prayed that it would not destroy my car and nor harm anyone else. Turns out that it busted part of my bumper and my front turn indicator.  At least no one got hurt and I had ended-up pushing it off the road.

What would a self-driving car have done?  We need to make sure that self-driving cars have the appropriate smarts to make these kinds of life-and-death decisions. In today’s column I’ve focused on a single piece of debris. That’s the easy example. Suppose there are multiple pieces of debris, and essentially the road is like an obstacle course, or maybe you might think of it as a minefield.  One time, I was driving along, minding my own business, when all of a sudden a truck ahead of me dropped a half dozen cans of white paint onto the freeway. I rolled over the cans, but they had already punctured and there was white paint splashing all about.  I carefully exited from the freeway to see what might have happened to the underbelly of my car. When I stopped at a gas station, I laid on the ground to look under the car. I was now the proud owner of a car that had white paint splashed all over the underbelly of the car. Fortunately, it did not seem to have caused any harm.  I was also lucky that the white paint could not be seen while standing and looking at the car (my car was all black in color, the white would have shown easily).

Roadway debris is a common occurrence and AI self-driving cars need to have the cognitive capabilities involved in ascertaining what to do about debris.  A one-size-fits-all approach of for example always hitting debris or always avoiding debris is not very sound. There are circumstances where hitting the debris is prudent, while in other cases hitting it would be a terrible choice. Likewise, there are circumstances that dictate avoiding the debris is the best option. Whichever it is, we are going to rely on our self-driving cars to figure this out.  For those self-driving cars that are less than the topmost Level 5, some car makers are assuming that the human driver will figure out what needs to be done. As I have repeated stated in many of my columns, the notion of suddenly handing control back to the human and have them make a split-second decision is risky and outright dangerous. AI developers and the auto makers need to ensure that self-driving cars have roadway debris acumen.  I am looking for my self-driving car to be roadway savvy, and so should you.

This content is original to AI Trends.