Autonomous Navigation Challenges, Part 2

In our last post we looked at the current state of the art in autonomous vehicle navigation. Another way in which the problem of navigation in unmapped or incorrectly mapped areas will be overcome is through artificial intelligence. We looked at the potential of this technology in our 4/10/17 post entitled, “Machine Learning and Work Zones”. Michael Reser published an article May 8th in Electronic Design entitled, “How AI Will Help Pave the Way to Autonomous Driving”.

Mr. Reser’s main point is that given the unfathomable quantity of data that must be digested and acted upon by autonomous vehicles (AVs) the technology will progress much faster and more accurately through machine learning. “Translating it all into a real-world challenge for AI-backed autonomous-driving systems, the expected outcome of such massive data processing is nothing short of getting the right answer in the shortest possible time to determine a proper action to avoid a traffic incident.”

“To put it differently, (a) large set of data in combination with realistic scenarios and nonlinear parameter sets enables systems and applications to fail safely and learn faster.”

He goes on to list the many challenges that must also be addressed including how to tie images from multiple sensors with varying resolution quality into one accurate picture. Another was how to validate and tie different data sources together in time. They must have a consistent way of labeling those sources in time.

Mr. Reser goes on to say they are not there yet, but he sees the process as inevitable.

“For true enablement of Level 4 and Level 5 automated driving, the system should be functional in all weather and driving conditions, which is obviously a given requirement. Still, it’s a much bigger challenge than sometimes mentioned and admitted”.

Like most AV challenges, this one has serious implications for work zones. It will be interesting to watch as this process unfolds.

One View of the Current State of the Art in Autonomous Navigation

Much has been written about autonomous vehicles and their methods of navigation. But most of that writing is little more than science fiction. The systems described are usually just concepts that engineers are working toward. What is the current state of the art?

Dyllan Furness posted May 9th about emerging technology in Digital Trends magazine His article, titled “Get lost: MIT’s self-driving car takes on unmarked roads” examined the current capabilities of autonomous vehicles. He found that current AVs are only able to drive on well-mapped city streets. This deficiency would affect autonomous vehicles ability to navigate a work zone as well. As he wrote in his opening lines, “If you find yourself on a country road in a self-driving car, chances are you’re both pretty lost. Today’s most advanced autonomous driving systems rely on maps that have been carefully detailed and characterized in advanced. That means the millions of miles of unpaved roads in the United States are effectively off-limits for autonomous vehicles.”

MIT is working to change that by developing a method of navigating using simple GPS, Google map data and a variety of sensors. ““We were realizing how limited today’s self-driving cars are in terms of where they can actually drive,” Teddy Ort, an MIT CSAIL graduate student who worked on the project, told Digital Trends. “Companies like Google only test in big cities where they’ve labeled the exact positions of things like lanes and stop signs. These same cars wouldn’t have success on roads that are unpaved, unlit, or unreliably marked. This is a problem.””

Certainly, work zones fall into this problem area. And MIT’s new system could address our issues, as well. In particular, by using Google map data this system would also pick up near real-time work zone data like we described in our 9/25/17 post. Then the sensors could identify traffic control devices and follow them safely through the work zone.

It is good to see that at least one organization understands the limits of current technology and is looking for a better, safer way for autonomous vehicles to find their way through rural roads and work zones.

Work Zone Traffic Control “Down-Under”

We just returned from a wonderful trip to Australia where we spoke to the Traffic Management Association of Australia (TMAA) about work zone ITS. Their members were all excited and focused on finding safer, more efficient ways to manage their work zones.

The program was packed full of interesting speakers and a variety of timely topics. They also gave us all just the right amount of time to discuss those topics between sessions. It was very well run.

The attendees seemed to enjoy talking to Americans and all asked what we thought of the meeting. My first answer was always the same: traffic control companies in both countries share the exact same set of problems:

1) Speeding in work zones.

2) End-of-queue crashes.

3) Hiring, training and retaining good employees.

4) A perception by the driving public that we are there to make their lives miserable.

5) Insufficient funding for maintenance and construction.

6) Changing standards and levels of enforcement from one state to the next.

7) Varying commitment and funding levels from one state to the next.

Just like ATSSA, the TMAA brings contractors, manufacturers, academia and government agencies together to discuss these problems and identify solutions. The TMAA does an especially good job of this. We look forward to learning more from them in the years to come!

Data Latency and Work Zone ITS

We met recently with a large local agency to discuss the idea of connected work zones and the concept of reporting work zones in real time to the digital maps we all use to get from Point A to Point B. She was excited about the idea but had concerns about delays that are sometimes experienced between the time when an incident occurs and the time when it is reported to you by your navigation app.

According to Waze, 65 million drivers regularly use their navigation service to get home as quickly and efficiently as possible. Drivers want to know about problems along their routes before they reach them and in time to take another faster route if it makes sense to do so. Richard Russell, a former sales engineer with Google, said five years ago that, “we actually want negative latency, and will perceive anything less as latency.”

That was about the time that Google purchased Waze. Waze works because users report problems in real time thus helping to reduce latency. HERE has found another way to reduce latency. They look at in-vehicle sensors such as hard braking sensors to identify and locate traffic issues the moment they begin. HERE also plans to begin including user reports to get as close to real-time reporting as possible.

Today, work zones are the single largest cause of non-recurring congestion. So, if we could report work zones in real time (see Work Zone Reporting to Autonomous Vehicles – posted 9/25/18) it will make these services even more valuable. Imagine arrow boards equipped with a device to report location and display status every time it is turned on or off!

Yet how will these services process an unimaginable amount of data including location, date & time, type of incident, and some form of verification and get it to the user without at least some delay? That is a problem only Waze or HERE can answer. We can tell you they are working on it.

In the meantime, some small amount of latency (a few seconds to as much as a minute) is going to exist. But the service is still valuable. In today’s worst-case scenario Driver A leaves home and asks for the fastest route to work. The app recommends the best one based on conditions at that time. Perhaps moments earlier an arrow board was turned on when a contractor closed a lane along that route for maintenance work. A short time later the app reports that roadwork and reroutes Driver A along a now preferable route. The app still saves him time, just not quite as much time as it might have with instant knowledge of all work zones.

Zero latency is the goal. But let’s not allow the perfect to be the enemy of good.

USDOT Roundtable on Data for Automated Vehicle Safety

On December 7th of 2017 the USDOT convened an interesting group of stakeholders to discuss automated vehicle data needs. The goal was simply to better understand what will be needed, so we can all work in that same direction. Attendees included automakers, regulators, local agencies, privacy advocates, data aggregators including Waze and HERE, universities, and industry.

They have published a short document detailing their findings. Download “roundtable-data-automated-vehicle-safety-report[3585]” here.

A set of four principles was discussed and supported by the group. Those included

  • Promote best practices for data security and privacy.
  • Act as a facilitator to promote voluntary data exchanges.
  • Start out small to find what works and then build on that.
  • Coordinate across modes to save time and money.

Number 2 is perhaps the most problematic. Vehicle and component manufacturers are still playing their cards very close to their vests. They will continue to protect whatever competitive advantage they feel they have. They don’t mind sharing what everyone else is sharing but don’t want to go beyond that point for obvious reasons. So, what will be shared will start with basics such as crash data, AV hours driven, etc. and will grow from there.

The good news, for our purposes here, is the discussion of high priority use cases. #1 on the list is “Monitoring Planned and Unplanned Work Zones”. The data they felt was of the highest value included, “Work zone locations, planned duration of project, updates, planned lane closures, changes in signing, directions, or parking.”

Other encouraging use cases include #2 “Providing Real-Time Road Conditions”. There they discuss the need for data on detours and missing or deficient signs and pavement markings.

Under testing discussions, there was an emphasis on safety-critical scenarios which would have to include work zones. Clearly manufacturers must test not just in ideal conditions, but in all conditions including bad weather, poorly delineated work zones, and in and around major and minor incidents.

They coined the term “Edge Cases” which refer to a “problem or situation that occurs only at the extreme operating parameter.” Certainly, most testing today will continue at or below 35 MPH on a sunny day and under controlled conditions. But once we are all satisfied that AVs can drive safety in ideal conditions, it will be time for the worst-case scenarios. Again, work zones will surely be a part of that.

The last use case of interest was improving roadway inventories. The group felt high-value data for this effort included,””edge-to-edge”, high-definition map elements (e.g., signs and signals, curbs, pavement markings, tolls, express lanes, bridge heights and weight capacities, highway dividers, overpasses, pedestrian areas, bicycle lanes, taxi drop-off zones, (and) quality metrics.”

Under “proposed federal roles” they talk about the USDOT acting as a facilitator of sharing and discussions between the various stakeholders. It’s good to know work zones are now a part of that discussion. Thank you to USDOT for helping make that happen. Our greatest fear just a few short years ago was that the automotive industry would get too far down the road with their development to accommodate special circumstances including work zones, special events and incident response. It’s great to see that won’t be the case.

Are Autonomous Vehicles Safe?

On February 6th we sat in on a FHWA T3 webinar entitled “Are Autonomous Vehicles Safe?” It was moderated by Dr. Francesca Favaro of San Jose State’s Mineta Transportation Institute. She runs a program known as RISA2S – “Risk & Safety Assessment of Autonomous Systems” and recently examined California DMV data from 2014 through 2016 on crashes and disengagements of automated vehicles.

Because their data is from automated vehicles on the road today, their focus has been on SAE Level 3 automation. That is, by and large, what is being tested now. Most manufacturers are not planning to offer Level 3 vehicles to the general public. But as a result their findings point out the strengths and weaknesses of Level 3 automation. So for that reason most of their presentation came to focus on disengagements – when the vehicle gives control back to the driver – and driver reaction time to those disengagements.

This is an issue of critical importance to work zones. At some point in the future autonomous vehicles will negotiate work zones without need of human input. But that is many, many years from now. In the meantime, a mixed fleet of cars and trucks with varying levels of automation will be passing through our work zones. As Paul Carlson said at the recent ATSSA Innovation Council meeting, “We have had a mixed fleet for some time now. Any discussions of a mixed fleet now are just the next iteration of that.”

So the two related issues of when AVs disengage control, and how drivers react to those disengagements will be an important point of discussion for the foreseeable future.

Their data on disengagements was very interesting. The frequency of disengagements is declining. In fact in 2016 they were one-third of what they were per mile travelled in 2014. So the technology is improving rapidly. Machine learning will continue that trend.

11% of all disengagements were due to external conditions. 49% were due to system failures of some kind. 33% were due to human factors. And 7% to “other”.

Work zones fall within external conditions. 2.22% of all disengagements were due to construction zones and 4.63% were due to poorly marked lanes. Now, we don’t know much about the conditions when and where this testing took place. Speeds were all at or below 30 MPH. But we don’t know if it was dry and sunny, or raining at night. And we don’t know if the time spent in work zones was typical of your average driver.

Chances are they were not. In fact, chances are current testing avoids work zones most of the time so disengagements in real-world work zones would likely be many times greater.

30.12% of disengagements are due to human factors – usually driver discomfort. Some of that is a trust issue with the technology. But a large part is a desire to stay safe in fast changing situations such as near incidents, work zones, or other higher volume conditions. So I think we need to include some of that data as well.

Next let’s consider driver reaction to these disengagements. This is really two issues: what the driver does, and how fast he or she does it. But the study looked only at driver reaction time. The California DMV has not yet defined what is meant by “reaction time” yet manufacturers are required to measure it. So the data presented is inconsistent in its terminology. With that in mind it is still interesting and helps point the way for future studies.

This testing was done in driving simulators. Subjects reaction times varied from 0.87 seconds to as much as 3.17 seconds. Again, that is in a simulator when the test subject was told to expect something. Other studies have shown mean reaction times of 2 seconds or more.

Let’s use the 2 second number, though it may be much higher. At 65 MPH a vehicle would travel nearly 200 feet in that time. A lot can happen in 200 feet when approaching or adjacent to a work zone. So clearly disengagements should occur in advance of the work zone giving drivers time to acclimate themselves to the situation around them. For the next 20+ years many of the vehicles passing through work zones will turn control over to the driver. So rather than try to deal with those within the confines of the work zone, lets’ just automatically turn control over to drivers before they enter it. That will provide a more predictable hand-off in a safer environment.

This study was just their first attempt. As more data is collected and as terminology becomes better defined and as testing ventures into more “real world” scenarios, we will learn much more. We look forward to future reports from RISA2S and other AV research organizations.

FHWA Request for Information Regarding Automated Driving Systems

This blog just posted a couple of days ago, but this topic can’t wait, so we are posting again today. On January 18th the Federal Register published a Request For Information from FHWA regarding automated driving systems (ADS). It asks DOT’s, manufacturers, trade associations, and other interested parties for their answers to questions in ten specific areas.  Download Docket FHWA-2017-0049. All comments must be received by March 5th, so please get started.

The work zone ITS world should be most interested in questions 4, 5, 9 and 10. You will also be interested in many of the other questions if you are involved in static signs, pavement marking or other permanent roadway safety infrastructure.

First, let’s commend the FHWA for asking these questions. It wasn’t long ago that we worried that automakers and regulating agencies did not know what they did not know. A great deal of time and effort has gone into building coalitions with these folks and it appears that is beginning to pay off.

So lets’ reward their curiosity with well written and timely responses. You can do so individually and/or through your industry associations such as the American Traffic Safety Services Association.

We will start the ball rolling now with these suggested responses to the four questions mentioned earlier. These are short in the interest of time, but should be fleshed out in any formal response.

Question 4: How should FHWA engage with industry and automation technology developers to understand potential infrastructure requirements?

Include work zone ITS providers and trade associations like ATSSA in the conversation. Many of us are already participating in AV events including the Automated Vehicles Symposium, the ITS World Congress, etc. Presentations at these events focused on work zones have lead to many productive conversations and several “ah-ha” moments for AV manufacturers. Please encourage more of this going forward.

Question 5: What is the role of digital infrastructure and data in enabling needed information exchange between ADS and roadside infrastructure?

Work zones will be the most common anomaly in digital maps unless we begin preparing now. Arrow boards, flagger stations, and other active work zones can be equipped to report their location and other pertinent information automatically and in real time. The same could be done for emergency responders and special events. Only by doing so can we hope to prevent crashes like the one reported January 23rd  ( Mercury News ). Safety is the driving force behind this initiative, but it will also reduce drive times, reduce air pollution, and improve the efficiency of our road networks.

Question 9: What variable information or data would ADS benefit from obtaining and how should that data be best obtained?

Work zones are the single most frequent cause of non-recurring congestion. Clearly real-time work zone data should be included. This data must include the precise location of the work zone, when and where lanes are closed and when they are opened up again, where flagging operations are taking place, and any other important features of the traffic control including lane splits, narrow lanes, crossovers, and full closures and detours. Finally delay times should also be included whenever available.

Question 10: What issues do road owners and operators need to consider in terms of infrastructure modifications and traffic operations as they encounter a mixed vehicle fleet during the transition period to a potentially fully automated fleet?

It is estimated that it will take at least 20 years to reach a point where the vast majority of vehicles are highly automated. Until then work zones will be just as dangerous for conventional vehicles as they are now. Don’t scrimp on traffic control devices, signage or pavement markings in the hopes that they won’t be needed.

As for those automated vehicles, most will not be capable of navigating autonomously though an active work zone. Plan to trigger the vehicles to hand over control to the driver well in advance of the work zone. Studies show drivers need a minimum of 8 to 10 seconds to regain situational awareness.

Work zones, incident response, and special events will test these systems more than anything else. Make them a big part of the conversation now to avoid problems in the not too distant future!