By Dr. Lance B. Eliot, the AI Trends Insider
Where are you, right now?
Are you standing up, sitting down, or laying down? Where were you thirty seconds ago? Where were you thirty minutes ago? Presumably, you are able to answer these questions. You should be able to. Your mind and body are keeping track of your position and movements, doing so without you necessarily being fully aware that it is taking place. Sometimes, such as if you stand-up and try to balance on one leg, you suddenly are aware of your body status, leaning slightly one way and then back the other way. This ability to keep track of your body and where you are is a standard feature of our bodies and involves a close coupling between aspects of your anatomy and your brain.
There’s actually a special word for this, it’s called your proprioception, and you have proprioceptors that do this work for you. Most people don’t realize it’s a thing. It is a thing. It’s an important thing. You need this to keep track of your physical navigation in the world.
And, it’s something that self-driving cars need to have too.
At the Cybernetic Self-Driving Car Institute, we’ve been developing and expanding the ability of self-driving cars to include proprioception. For humans, we use bodily information that comes from your muscles, your tendons, and other internal organs to keep track of your movement and position (along with your brain, of course). For self-driving cars, the device that provides that same kind of tracking of movement and position is called an Inertial Measurement Unit (IMU). Any bona fide self-driving car has one.
There are various sensory devices on self-driving cars (see my article on sensor fusion for self-driving cars). The most obvious sensors are those such as radar to detect objects, LIDAR to likewise detect objects, sonar for tracking objects, etc. Since we’re now using a big word in proprioception, let’s add to our vocabulary and we’ll refer to those sensors such as radar, LIDAR, sonar as exteroceptors. For humans, exteroceptors are your nose, ears, eyes, and the like, which have to do with sensing information that originates outside of your human body. That’s what radar, LIDAR, and sonar do for a self-driving car, they sense information outside of the car.
The IMU is a proprioceptor that keeps track of where the self-driving car is, and does so by movement of the self-driving car (it is considered “inside” info versus the exteroceptors that are collecting “outside” info). You might say to me, Lance, we already have something to readily keep track of a car, namely a GPS system. Yes, a Global Positioning System (GPS) is indeed another device used by a self-driving car. It is another essential device, for sure. Using access to the Global Navigation Satellite System (GNSS), you can keep track generally of where your car is. We use this in everyday cars. We use this in our cell phones and have it so we can pull up a map and see where we are. It is used extensively by ridesharing services such as Uber, showing you where you are and where your ride is.
But have you ever had your GPS either not work or work in a spotty way? You must have. I am sure you’ve experienced circumstances wherein you said something like “I am getting a lousy signal” and your GPS was wrong or inaccurate. I was in downtown San Francisco the other day on a quick work trip there, and when I tried to use Uber (this is not an advertisement for them, use whichever ridesharing service you like!), it turned out that the driver thought I was a block away from where I was actually standing. We normally expect our GPS to be within a few feet or meters of accuracy. In this case, it was nearly a city block off.
This makes sense that the GPS would be a kilter. I was standing in a dense downtown area and surrounding me were very tall skyscrapers. The signals for GPS come from orbiting satellites and so any dense object can block or distort the signals. Skyscrapers are certainly highly dense objects, usually composed of masses of metal and steel. Another place that you probably have experienced difficulties using your GPS would be inside a tunnel. If you go into a subway system, bringing up your GPS will usually show a very generalized indication of your location and not be especially accurate.
Believe it or not, your GPS can be distorted by other aspects such as trees. What, you ask, trees? Yes, if you are in a dense wooded area, the trees can also impact the reception of the signals from the satellites. Even clouds and rain can also impact your GPS. Not only can all of these conditions block the signals, it can allow them to pass but cause them to be delayed. You might not have noticed this. If you watch your GPS for a few minutes continuously, you will sometimes notice that it is tracking you step by step, but then seems to jump a step. This is due to a delay as a result of some object that momentarily blocked the signal.
The point to this lengthy lesson about GPS is that simply stated the GPS in your car is not entirely reliable for purposes of knowing where the car is. The GPS signals can be blocked entirely or delayed in their arrival. If you are walking on the street, you might be satisfied with a momentary blockage or delay that’s a second or two, or maybe a fraction of a second. If you’re in a car, and the car is traveling at 80 miles per hour, do you know how many feet your car travels in one second? Take a guess. Ready for the answer? Your car travels about 117 feet.
Traveling 117 feet per second, and not knowing where your car is because the GPS is blocked or delayed, it’s not much of a problem when you as a human are driving a car. You are still aware of what is going on around the car. But, for a self-driving car, not knowing where the car is for 117 feet because there has been a GPS delay is a really bad thing. The virtual model that the AI is keeping of where the self-driving car is, and where other objects are, and where those objects are headed, and where the self-driving car is headed, all of those can get out of sync.
This could be so bad that your self-driving car could get itself into an accident. In essence, it will have lost its way in space and time, for a substantive distance and time, and the ability of the sensors and the AI to deal with that lost perception of body and movement is something we want to try and prevent from happening.
That’s why we need an IMU.
An Inertial Measurement Unit (IMU) is a device that is mounted in a fixed position within the body of your self-driving car. It needs to be in a fixed position of the car, so that it is aware of where its anchored placement is. From this fixed position within the car, the IMU will keep track of the movement and position of the car. Usually consisting of several gyroscopes and accelerometers, the IMU provides to the AI of the car an indication of the pitch, roll, and heading of the self-driving car (these are referred to as rotational parameters as sourced by the gyroscopes), and also tracks the linear acceleration of the vehicle (that’s by the accelerometers).
The reason there are several gyroscopes and several accelerometers in the IMU is to ensure redundancy. Let’s suppose we have an IMU that contains three gyroscopes and three accelerometers. We can ask each gyroscope what it is sensing, and compare them to ensure they are all consistent. If one of them differs, we could infer that it has somehow gotten out of tune or maybe is broken. We could then rely on just the other two gyroscopes. Same story for the accelerometers. We have each accelerometer telling what it knows, and if there seems to be a problem with one of them then we have the other two as a failsafe form of redundancy.
You might be thinking that if the self-driving car has an IMU then maybe it does not need the GPS. Why have both, you ask? Having both is crucial. They work hand-in-hand. The IMU can only tell you relative position and movement. The GPS tells you so-called absolute position and movement. What’s the difference? The GPS is saying where you specifically are. The IMU tells you how far you’ve moved from some other position.
For example, the GPS tells me that I am physically standing in Disneyland at coordinates of a certain latitude and longitude. With my eyes, I can see that I am standing near the statue of Walt Disney that resides in front of the Sleeping Beauty Castle. For the IMU, if I had been originally standing at the Walt Disney statue, and I had then walked away from it, the IMU would be able to have indicated that I had walked at a certain angle from that starting position and that I had gone a certain distance which can be calculated based on parameters associated with me and the IMU.
The IMU cannot tell the self-driving car where the self-driving car is per se. Instead, the IMU can tell the self-driving car what has happened in terms of movement and position on a relative basis from some starting point. The IMU is providing the angular velocity and linear acceleration, which can be used to calculate where I am, relative to where I was. For those of you that have ever gone camping with the Boy Scouts or Girl Scouts, you might know this as dead reckoning. Or, if you are sailor, you certainly know about dead reckoning. Dead reckoning is the act of identifying a known starting location and then keeping track of where you have gone over time as relative to that starting position.
In the woods, you might tell the scouts to set as a starting point a large boulder. Then, you ask them to walk a distance away from the boulder, but as they do so, keep track of the angle that they have walked and how many paces they walked. Based on their paces, you would have previously measured their gait, suppose they go a distance of three feet for each step, you could then calculate where you are now, relative to where the boulder is.
You need to have some good starting position for the use of dead reckoning. For the IMU in your self-driving car, it will use the GPS to provide that starting position. Let’s suppose the GPS says that your self-driving car is at a stated latitude and longitude. The IMU uses that as a stated starting position. The self-driving car then drives for a few seconds. The IMU is keeping track of the rotational parameters and the acceleration. The AI of the self-driving car is getting this fed to it by the IMU. Based on the data from the IMU, the AI can update the virtual model of where the self-driving car is.
The IMU is providing this relative position data continuously. Anyone that has ever done dead reckoning knows that one of the pitfalls of dead reckoning is that there can be errors that accumulate over time and so your relative position can become murky. For the scouts, I mentioned that we’d keep track of their number of steps and then use that to calculate the distance walked. Each step is not exactly the same amount of distance. Thus, when we multiply out the number of steps times what we assume the average step length to be, we are introducing an error since this is only an estimate of the actual distance gone.
The IMU and the AI might be using the circumference of the wheels of the car, along with the number of wheel rotations, in order to estimate the distance traveled. This is equivalent to using the number of steps of the scouts and an estimate of how far they travel with each step. The IMU uses some kind of predetermined algorithm to help translate its gyroscope and accelerometer readings into something that can be used by the AI to ascertain the position of the self-driving car.
I had stated that the GPS and the IMU work hand-in-hand. Let’s see why this is indeed the case. The GPS says the self-driving car is at a stated latitude and longitude. Assume the GPS signal is strong and so the AI believes that the latitude and longitude is correctly indicated. The self-driving car moves ahead as directed by the AI. During the next say two seconds, we don’t have a GPS signal because it is blocked by tall buildings. Meanwhile, the IMU has been continuously reporting in real-time. The AI is updating the virtual model and believes that the self-driving car is now at position X and Y, relative to the latitude and longitude that it had earmarked two seconds ago.
As you can see, the self-driving car knows where it is, even though the GPS has been blocked for two seconds. While the GPS was being blocked, the IMU was indicating where the self-driving car has moved on a relative position from the “starting point” of the last reliably reported latitude and longitude. Let’s pretend that after those two seconds has elapsed, the GPS now reports the latest latitude and longitude and that the signal is strong and reliable. The AI can then update itself and see how far afield it is from where it thought it was. We now have also a new “starting position” since the latest GPS coordinates can be used instead of the coordinates from two seconds ago.
Therefore, the GPS is augmented by what the IMU has to say, and when the GPS is not available the IMU is still allowing us to keep track of where the self-driving car is. In fact, the IMU can be used to always try to verify that the GPS is being accurate, since you could be double-checking the GPS against whatever the IMU has been saying. If we decide that a starting position is when we begin our driving journey, we could be collecting the IMU data throughout the trip and be using it to always try and double-check what the GPS is stating. Of course, we need to keep in mind the “errors” that accumulate when using dead reckoning, and over any substantive length of time and distance those will be relatively large.
Another purpose of the IMU is to allow the AI to determine the slip angle of the self-driving car. The slip angle is an indication of whether the car is maybe skidding, maybe spinning, or maybe tending toward rolling over. The IMU provides the rolling wheel actual direction versus where the wheel is pointing. You’ve likely experienced this aspect when driving on a slick road.
You might have the wheels pointed toward the left, but the car is actually moving or skidding toward the right. You’ve lost control of the steering since the tires don’t have sufficient traction on the road surface. Remember how they say that you should then turn the wheels in the direction of actual travel and try to regain control of the steering? That’s something that the AI needs to be aware of since it too is trying to drive the car as a human would.
The IMU provides to the AI a sense of whether the self-driving car is getting itself into a posture that needs special attention. Imagine you are driving a car and you come off the freeway at a high rate of speed. The off-ramp is curved. You’ve likely had that scary sensation that the car felt like it was going to tip over as you took that curve, because you were going faster than recommended and the physics of the car and the curve and the road and your speed are tending to tip over the car. As a human, you could feel that sensation of tipping.
The AI needs to be able to have that same sensation. It needs to know whether the car is maybe tipping over. If so, the AI would try to control the car to prevent the car from rolling over. The IMU provides the sensory information for the AI to figure this out. The IMU is pretty handy, since it can be used to keep track of location as done in conjunction with the GPS, and it can also be used to detect circumstances involving the car skidding, spinning, or tipping over. The IMU can be rather precise in its measurements, including within a centimeter of precision, such as tracking the velocity to a 2 centimeter per second accuracy level (or better).
Some like to refer to the IMU as the inner ear of a self-driving car. When the AI is doing sensor fusion, the GPS and the IMU are considered crucial to knowing where the car is, where the car has been, and where the car is headed. We all know about a GPS because we see it on our phones and on displays in our cars. Few know about the IMU. It is one of those silent “organs” deep within the car and that the AI needs to know about on behalf of the humans in the self-driving car and being able to drive the car for them.
For humans, we have our own IMU in that our muscles and tendons, working with our brain, telling us the same kind of relative position and movement information. Any AI developer that is doing systems for self-driving cars should be aware of the important dance between the GPS and the IMU. If you’ve never heard of the IMU, you might fall off your chair to know that it exists. Of course, your proprioceptors will let you know as you begin to tip off the chair and head to the floor. It won’t cushion the blow, but at least your mind can get ready for the floor, command your arms to catch you, and you’ll be less likely to get hurt. The hero here was the IMU!
This content is original to AI Trends.