USDOT Roundtable on Data for Automated Vehicle Safety

On December 7th of 2017 the USDOT convened an interesting group of stakeholders to discuss automated vehicle data needs. The goal was simply to better understand what will be needed, so we can all work in that same direction. Attendees included automakers, regulators, local agencies, privacy advocates, data aggregators including Waze and HERE, universities, and industry.

They have published a short document detailing their findings. Download “roundtable-data-automated-vehicle-safety-report[3585]” here.

A set of four principles was discussed and supported by the group. Those included

  • Promote best practices for data security and privacy.
  • Act as a facilitator to promote voluntary data exchanges.
  • Start out small to find what works and then build on that.
  • Coordinate across modes to save time and money.

Number 2 is perhaps the most problematic. Vehicle and component manufacturers are still playing their cards very close to their vests. They will continue to protect whatever competitive advantage they feel they have. They don’t mind sharing what everyone else is sharing but don’t want to go beyond that point for obvious reasons. So, what will be shared will start with basics such as crash data, AV hours driven, etc. and will grow from there.

The good news, for our purposes here, is the discussion of high priority use cases. #1 on the list is “Monitoring Planned and Unplanned Work Zones”. The data they felt was of the highest value included, “Work zone locations, planned duration of project, updates, planned lane closures, changes in signing, directions, or parking.”

Other encouraging use cases include #2 “Providing Real-Time Road Conditions”. There they discuss the need for data on detours and missing or deficient signs and pavement markings.

Under testing discussions, there was an emphasis on safety-critical scenarios which would have to include work zones. Clearly manufacturers must test not just in ideal conditions, but in all conditions including bad weather, poorly delineated work zones, and in and around major and minor incidents.

They coined the term “Edge Cases” which refer to a “problem or situation that occurs only at the extreme operating parameter.” Certainly, most testing today will continue at or below 35 MPH on a sunny day and under controlled conditions. But once we are all satisfied that AVs can drive safety in ideal conditions, it will be time for the worst-case scenarios. Again, work zones will surely be a part of that.

The last use case of interest was improving roadway inventories. The group felt high-value data for this effort included,””edge-to-edge”, high-definition map elements (e.g., signs and signals, curbs, pavement markings, tolls, express lanes, bridge heights and weight capacities, highway dividers, overpasses, pedestrian areas, bicycle lanes, taxi drop-off zones, (and) quality metrics.”

Under “proposed federal roles” they talk about the USDOT acting as a facilitator of sharing and discussions between the various stakeholders. It’s good to know work zones are now a part of that discussion. Thank you to USDOT for helping make that happen. Our greatest fear just a few short years ago was that the automotive industry would get too far down the road with their development to accommodate special circumstances including work zones, special events and incident response. It’s great to see that won’t be the case.

Are Autonomous Vehicles Safe?

On February 6th we sat in on a FHWA T3 webinar entitled “Are Autonomous Vehicles Safe?” It was moderated by Dr. Francesca Favaro of San Jose State’s Mineta Transportation Institute. She runs a program known as RISA2S – “Risk & Safety Assessment of Autonomous Systems” and recently examined California DMV data from 2014 through 2016 on crashes and disengagements of automated vehicles.

Because their data is from automated vehicles on the road today, their focus has been on SAE Level 3 automation. That is, by and large, what is being tested now. Most manufacturers are not planning to offer Level 3 vehicles to the general public. But as a result their findings point out the strengths and weaknesses of Level 3 automation. So for that reason most of their presentation came to focus on disengagements – when the vehicle gives control back to the driver – and driver reaction time to those disengagements.

This is an issue of critical importance to work zones. At some point in the future autonomous vehicles will negotiate work zones without need of human input. But that is many, many years from now. In the meantime, a mixed fleet of cars and trucks with varying levels of automation will be passing through our work zones. As Paul Carlson said at the recent ATSSA Innovation Council meeting, “We have had a mixed fleet for some time now. Any discussions of a mixed fleet now are just the next iteration of that.”

So the two related issues of when AVs disengage control, and how drivers react to those disengagements will be an important point of discussion for the foreseeable future.

Their data on disengagements was very interesting. The frequency of disengagements is declining. In fact in 2016 they were one-third of what they were per mile travelled in 2014. So the technology is improving rapidly. Machine learning will continue that trend.

11% of all disengagements were due to external conditions. 49% were due to system failures of some kind. 33% were due to human factors. And 7% to “other”.

Work zones fall within external conditions. 2.22% of all disengagements were due to construction zones and 4.63% were due to poorly marked lanes. Now, we don’t know much about the conditions when and where this testing took place. Speeds were all at or below 30 MPH. But we don’t know if it was dry and sunny, or raining at night. And we don’t know if the time spent in work zones was typical of your average driver.

Chances are they were not. In fact, chances are current testing avoids work zones most of the time so disengagements in real-world work zones would likely be many times greater.

30.12% of disengagements are due to human factors – usually driver discomfort. Some of that is a trust issue with the technology. But a large part is a desire to stay safe in fast changing situations such as near incidents, work zones, or other higher volume conditions. So I think we need to include some of that data as well.

Next let’s consider driver reaction to these disengagements. This is really two issues: what the driver does, and how fast he or she does it. But the study looked only at driver reaction time. The California DMV has not yet defined what is meant by “reaction time” yet manufacturers are required to measure it. So the data presented is inconsistent in its terminology. With that in mind it is still interesting and helps point the way for future studies.

This testing was done in driving simulators. Subjects reaction times varied from 0.87 seconds to as much as 3.17 seconds. Again, that is in a simulator when the test subject was told to expect something. Other studies have shown mean reaction times of 2 seconds or more.

Let’s use the 2 second number, though it may be much higher. At 65 MPH a vehicle would travel nearly 200 feet in that time. A lot can happen in 200 feet when approaching or adjacent to a work zone. So clearly disengagements should occur in advance of the work zone giving drivers time to acclimate themselves to the situation around them. For the next 20+ years many of the vehicles passing through work zones will turn control over to the driver. So rather than try to deal with those within the confines of the work zone, lets’ just automatically turn control over to drivers before they enter it. That will provide a more predictable hand-off in a safer environment.

This study was just their first attempt. As more data is collected and as terminology becomes better defined and as testing ventures into more “real world” scenarios, we will learn much more. We look forward to future reports from RISA2S and other AV research organizations.