Emergency Vehicle Awareness for Self-Driving Cars


By Dr. Lance B. Eliot, the AI Trends Insider

It used to be that when you heard the sound of a siren coming from a police car or ambulance or fire truck that all nearby drivers would scurry to pull their cars over to the side of the road. In California (and in most other locales in the United States), it’s the law that you must yield the right-of-way to any emergency vehicle that is using a siren or deploying flashing lights. You used to see drivers pull over religiously and without hesitation, but lament that it seems that today’s drivers are less apt to abide by this law. Some drivers will speed-up in hopes of somehow racing ahead of the emergency vehicle. Some drivers make a rather feeble attempt to pull over, as though they are quasi-acknowledging the law, but pretty much otherwise are determined to proceed and unwilling to relinquish the road to the emergency vehicle. Whether due to laziness, disdain, confusion, ignorance, or defiance, many drivers aren’t as rigorous as they once were about obeying this law.

Of course, these scofflaws are endangering all potential stakeholders, including endangering themselves, the passengers in their car, other drivers and their passengers, and the emergency vehicle with its occupants. By remaining in the midst of the roadway or taking bizarre actions, they can cause the emergency vehicle to have to swerve dangerously to avoid them. Sometimes the emergency vehicle will need to come to crawl to deal with those blocking traffic, and so this endangers too the lives of those that perhaps the emergency vehicle is trying to save, such as an ambulance that is rushing an injured patient to the hospital or a fire truck that is trying to get to a raging fire to rescue people trapped in a burning building.  I’ve asked some of these danger-producing drivers why they don’t seem to care about the lives of others and usually they feign innocence. They didn’t hear the siren. They didn’t see the flashing lights. They heard the siren but didn’t think it was anything worth being concerned about. They saw the flashing lights but figured the emergency vehicle would just find a way around their car and so they decided to just let the emergency vehicle driver worry about what to do.

I will get off my high-horse about these inhumane human drivers and now focus instead on the emerging AI-based self-driving cars. What would a self-driving car do when an emergency vehicle approaches?  By-and-large, nearly all of the existing self-driving cars are ill-equipped to detect, recognize, and react to an emergency vehicle. Since most of the self-driving cars are at a less than level 5 capability (see my piece about the Richter scale for self-driving cars levels), the self-driving car makers assume that the human driver in the self-driving car will be the one to detect that an emergency vehicle is nearby, and it will be up to the human driver to take over controls of the self-driving car.  This is problematic, as I’ve stated in many previous columns, due to the aspect that the human driver in the self-driving car might take over and not take the correct action due to a cognitive “gap” in what is occurring in the driving environment at the moment, or the human driver might not have time to properly react and thus belatedly take over control of the self-driving car.

Imagine if the human driver hears the siren of an approaching ambulance, which is nearing the car at a speed of 60 miles per hour and the car itself is going 30 miles per hour. The human driver detects the siren, mentally calculates that there is something going on nearby, looks around to see if the emergency vehicle can be seen, and suppose they suddenly see it rushing up behind them. Now, this human driver that is in a self-driving car and for which we’ll assume that the self-driving car is doing the driving, will need to mentally ascertain the status of their own car, what speed it is going, what direction, what is the other traffic around them, where is the safest spot to pull over, what will happen to other traffic when they try to pull over, and so on.  Next, the human driver will need to put their hands on the steering wheel and their feet onto the pedals of the car, and disengage the AI of self-driving car. Finally, the human driver will then need to take the evasive action.  All of this often must occur with just a few seconds of hearing the siren or seeing the flashing lights.

It is pretty easy to predict that we’re going to have problems when these kinds of situations arise. The human driver will be stressed to make the right decision and maybe make the wrong decision and radically pull to the side of the road.  But, this might cause a cascading problem for other drivers, and those cars now suddenly need to do risky maneuvers to get out of the way of the human driver and also of the approaching emergency vehicle.  This can become a “game” of bombardment with other cars bumping into each other or otherwise scrambling and scraping or hitting each other. Keep in mind too that the traffic will likely have a mixture of both human driven cars and self-driving cars.  This is crucial because it means that the AI of the self-driving cars needs to be able to contend with traffic that radically goes awry when an emergency vehicle situation presents itself.

In our Cybernetics Self-Driving Car Lab we have examined what happens in these kinds of situations.

One scenario involves the AI not knowing that an emergency vehicle is approaching and so is unaware of the circumstances and what is going to arise.  This lack of awareness is partially due to the fact that almost none of the self-driving car makers are outfitting their self-driving cars with an audio capturing capability, and thus the self-driving car has no chance to “hear” the sounds of sirens. We have been exploring the use of sound capturing equipment that would be listening for crucial sounds such as a siren. The self-driving cars also don’t tend to look for flashing lights that are on an emergency vehicle. We are exploring ways of visually detecting the flashing lights, via interpreting the visual images being captured by cameras on the self-driving car.  This is a harder problem than you might think. The emergency vehicle can be at a great distance and so hard to see, the flashing lights can be obscured by other objects and cars that are on the road, etc. If the AI cannot do an early detection about the approaching emergency vehicle, it hampers the chances of the AI being able to take appropriate action, and will again require the human driver to intervene.

Another scenario that comes to play is the reaction of the other cars around the self-driving car. Even if the self-driving car cannot hear the siren and nor detect the flashing lights, it can detect that other cars around it are making unusual maneuvers. In other words, imagine that you are driving a car and suddenly you notice the cars around you are all jockeying to get over to the side of the road. You hopefully are astute enough to deduce that something is happening.  Now, you might not be sure that those cars are maneuvering due to an emergency vehicle, and it might be that there is a mighty earthquake shaking the earth or an apocalypse taking place, but at least you are sparked into the realization that something is happening and you should likewise probably do something about it.

The AI of the self-driving car in its simplest form has to figure out how to yield right-of-way to the approaching emergency vehicle. This is the easiest of the steps. What requires even more intellectual punch is to deal with the rapidly evolving situation of other cars also scrambling to get out of the way of the emergency vehicle.  In other words, if you only anticipate that your car needs to get over, and if you assume that all other cars stay where they are, that’s an unrealistic way to program the self-driving car. You need to assume that all other traffic will be doing something, whether it is an attempt to react to the emergency vehicle or whether it is ignoring or not responding to the emergency vehicle.  All circumstances are feasible, involving a mix of both human drivers and self-driving cars, and some that will be reacting dramatically, some that will be reacting mildly, and some that aren’t reacting at all.

I have heard some self-driving car makers claim that this act of responding to emergency vehicles is a “non-problem” because they are assuming that soon all vehicles will be equipped with V2V (vehicle-to-vehicle communications), and therefore they are anticipating that the emergency vehicle will transmit a message to the self-driving car that says “get out of the way.” I agree that eventually we’ll have this kind of communications. It isn’t going to be anytime soon. The utopian world of these self-driving car makers is decades away.  Not only do the self-driving cars need to get equipped with these V2V capabilities, there will need to be standard protocols established and promulgated about what those messages are and how to convey them.  Different self-driving car makers will likely craft their own protocols. The regulatory bodies will eventually weigh-in.  Meanwhile, there is also the chance of someone spoofing these protocols and tricking your self-driving car into getting out of the way when they instead are rushing to that nighttime baseball game, and not due to being an emergency vehicle being in an actual emergency situation (see my piece on cybersecurity for self-driving cars).

I have been herein simplifying the situations too by describing the situations as merely getting over to the side of the road when reacting to the emergency vehicle.  There are a myriad of other circumstances that need to be considered. For example, one day I was driving on the freeway and a highway patrol car with a siren and flashing lights was in the far left lane of traffic and trying to get ahead of a pack of cars (I was driving in that pack).  The highway patrol car came up to where I was driving, and then pulled in front of the cars at my lateral position, and began to weave back-and-forth across all lanes of traffic. If you’ve never seen this happen, please be aware that this a driving tactic that means that all traffic is to stay behind the highway patrol car.  This is something too that we have been teaching our AI self-driving car system to recognize and react to.

Anyway, the highway patrol car starts weaving. Besides having us stay behind his car, the patrolman also began to go slower and slower.  This led to all of the traffic to go slower and slower too.  Again, this is a common trick, and is being done to create a break in traffic up ahead of the patrol car.  Usually, there is something that has landed onto the roadway, some kind of debris, and another patrolman up ahead wants to get out of his car and grab the debris.  The patrolman slowing down traffic is creating a break in traffic that will ensure that no cars are speeding toward the other patrolman and he or she can then run out into the freeway to quickly retrieve the debris.

Well, what happened next can be somewhat shocking if you’ve not seen this happen before. The highway patrol car weaves nearly to a stop, and so I also was coming to a near stop, and then all of sudden the patrolman makes a U-turn in the middle of the freeway and faces the oncoming traffic (which has now come to a complete stop). It was like a gunfighter facing a crowd of gunfighters. Face to face.  Somewhat eerie and disconcerting. Had no idea what this was about.  A few minutes later, a helicopter appeared overhead. Turns out that they wanted to land the helicopter onto the freeway, doing so to place an injured person into the helicopter and be whisked away to a hospital.  Though I was upset that I had to wait and was greatly delayed in getting to work, it is admittedly a rare occasion that you get to see a helicopter land on the freeway.  A sight to behold. Naturally, I also was hoping that the injured person would urgently get their needed medical care.

The action of making the U-turn was pretty unusual, though it can and does happen. The weaving and slowing of traffic happens routinely. These are all scenarios that a self-driving car needs to be able to recognize and then intelligently react to. Think of the many other circumstances that can also arise. For example, a motorcade of the mayor or the president, which then requires cars to appropriately act. A funeral procession. A tow truck that is towing a disabled vehicle. A highway patrol car that is pulling over a driver to give them a ticket. A car chase of a wild driver trying to get away from the police (as trivia, you might find of interest that the Southern California area is somewhat considered the “king” of car chases and we have them all the time; the news media loves it and will switch all TV and radio programming over to watching the car chase unfold).

The AI of the self-driving car needs to cope with these situations. We aren’t going to have V2V widespread any time soon.  And, relying solely on the back-up human driver to take the controls of the self-driving car is a lousy approach.  Plus, we’ll never get to the true self-driving car, the level 5, until we are able to get the AI to handle the emergency vehicles circumstances.  There are other twists too, such as you are not allowed to stop your car in an intersection when trying to react to an emergency vehicle, and so this is another rule that the self-driving car needs to know about.  Also, emergency vehicles can and will often use the wrong side of the road to make their way and avoid being slowed down by other cars.  This is another tactic that the self-driving car needs to take into account.

On the topic of detecting audio, besides detecting a siren, the self-driving car needs to ascertain how far away the siren is.  Is the siren getting closer or going further away? What kind of siren is it? Is it truly of an emergency vehicle or some other kind of siren (notably, where I live, each month they run sirens all around town as part of the monthly tsunami warning system, which even naive human drivers sometimes mistake as an emergency vehicle sounding its alarm).  Detecting a siren is more than merely picking the sound out of the cacophony of street sounds. It also requires intelligent analysis. Then, there are other potential sounds too, such as a patrol officer using their loud speaker.  Just the other day, I had a patrol officer in their car that was telling cars ahead of it to slow down.  In this case, imagine the complexity of trying to hear the spoken words, interpret the spoken words, assess that they are legitimate commands by an authorized officer of the law, and then act upon what is being spoken.  This is many times harder than merely detecting just a siren.

There is very little effort as yet on having AI that can detect, recognize, and respond to emergency vehicles. In our Cybernetics Self-Driving Car Lab, we are pushing the envelope on AI for self-driving cars by establishing system tactics and strategies to cope with emergency vehicles.  Using simulations and machine learning, we are improving self-driving car capabilities so that back-up human drivers won’t be thrust into the dire throes of having to figure out at the last minute how to take over controls of the car, and also so that we can be heading toward the level 5 self-driving car that can handle these situations all by itself.  Next time that you are driving and encounter an emergency vehicle, be thinking about the thinking that is required to deal with an emergency vehicle.  Plus, be aware of how complex the situations can be, and how deadly, and also how life saving those moments are, and the crucial nature of intelligent reactions to them.  Drive safely out there!

This content is original to AI Trends.