Tesla’s high-flying image, which had been moving from strength to strength since early 2013, hit its biggest speed bump last year when its Autopilot semi-autonomous/Advanced Driver Assist System (ADAS) came under scrutiny in the wake of Joshua Brown’s death. Suddenly Tesla’s pioneering Autopilot system went from being one of the company’s key strengths to being a serious liability that raised troubling questions about the company’s safety culture. Tesla CEO Elon Musk tried to swat away these concerns with what proved to be a set of highly misleading statistics about Autopilot safety, but the issue was not laid to rest until NHTSA closed its investigation with a report that seemed to exonerate Autopilot as a safety risk. With a single sentence, NHTSA shut down the most dangerous PR problem in Tesla’s history:
The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation.
Because NHTSA is the federal authority on automotive safety, with unparalleled resources to assess and investigate safety risks, this single sentence effectively shut down public concerns about Autopilot’s safety. In a terse statement on its company blog, Tesla noted
we appreciate the thoroughness of NHTSA’s report and its conclusion
But how thorough was NHTSA’s investigation, and how accurate was its conclusion? As it turns out, the questions around Autopilot’s safety may not be as settled as Tesla and NHTSA would have you believe.
The heart of all of Autopilot’s problems has always been the same: inconsistent company communication about what Autopilot truly is, resulting in widespread confusion about what the system really is and how it works. Though Musk and Tesla have sold Autopilot as a single system, which they claim is comparable to an aircraft autopilot system, it is in fact a suite of sub-systems and functions whose identities have been consciously blended together so that the overall system resembles a “self driving car” product. It appears that even NHTSA did not fully consider the many aspects of the Autopilot system in their assessment of its safety performance, which may have resulted in a highly misleading public conclusion in their report. In order to understand how this happened, we must break Tesla’s safety systems into at least four of its constituent parts.
There are two relevant systems that are not considered part of Autopilot:
- Forward Collision Warning (FCW), which senses obstacles and warns the driver when a collision may become likely.
- Automated Emergency Braking (AEB), which automatically mitigates collision impact by braking when a collision is imminent.
There are also two systems that make up Autopilot:
- Tesla Adaptive Cruise Control (TACC), which maintains vehicle speed and matches it to forward traffic.
- Autosteer, which is an advanced lane-keeping function that allows the vehicle to steer itself.
Of these four systems, Autosteer is the most unique to Tesla. FCW, AEB and TACC have all been available on a number of other vehicles for some time and none of them are ever confused with “self driving” or even semi-autonomous functions. Autosteer, which takes away human control of the steering wheel, is the key to Autopilot’s differentiated branding and immense popularity. Though other firms offer versions of lane keep assist, they are generally not considered to be as advanced as Tesla’s and (with the exception of Mercedes Benz’s DrivePilot) they are not marketed or communicated about as “self driving” or semi-autonomous. Thus it is of critical importance to Tesla’s marketing efforts that NHTSA stated specifically that
The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation. [emphasis added]
NHTSA was not making a judgement about the safety of Tesla’s Autopilot system or even its broader safety suite, which would include FCW and AEB. Instead it singled out Tesla’s most unique function, the function that allows Tesla to overhype Autopilot’s autonomous capabilities, as the key to the company’s most significant safety advance. But what if something else could explain that 40% improvement in crashes?
To make its 40% crash reduction claim, NHTSA looked at data from Model Year 2014, 2015 and 2016 Model S vehicles as well as Model Year 2016 Model X (NB: unlike most other manufacturers, Tesla’s Model Year designations match up with the year in which vehicles were actually manufactured). Autosteer was introduced to Teslas starting with the 7.0 software update in October of 2015, so NHTSA had nearly as much data from before the introduction of Autosteer as after. But there’s a complication that NHTSA fails to note anywhere in its report: in March of 2015, Tesla’s software update 6.2 introduced AEB to a safety suite that already included FCW.
This turns out to be a highly significant factor, as an IIHS study points out an amazing coincidence: though FCW alone has relatively little effect on the rate of collisions resulting in injuries (6%), adding AEB to FCW results in an improvement in this metric of just over 40%.
FCW with AEB reduced rear-end striking crash involvement rates [by 39%]… FCW with AEB reduced rates of rear-end striking crash involvements with injuries by 42% and rates of rear-end striking crash involvements with third-party injuries by 44%, but reductions with FCW alone were not statisticallysignificant (6% and 4%, respectively
In other words, the 40% improvement in crashes resulting in airbag deployments (an imperfect metric in itself, but that’s a debate for another day) that NHTSA ascribed to Autosteer seems far more likely to have been caused by the addition of AEB to FCW just months before the introduction of Autosteer. Though the NHTSA report references Tesla’s continuous updates to its safety suite after the introduction of Autosteer in October of 2015, it makes no reference to the introduction of AEB earlier that year or the potential impact that might have had on its statistical analysis. This strongly suggests that NHTSA did not control its statistical analysis of Autosteer safety for the addition of AEB, which just so happens to have an almost identical statistical impact on safety as the one NHTSA ascribed to Autosteer.
This is an extremely troubling revelation, with only two possible conclusions. One possibility is tht NHTSA was simply not familiar with IIHS’s highly relevant AEB research, which came out in January of 2016, suggesting deep incompetence on the part of America’s auto safety regulators. The other, even more bothersome possibility, is that NHTSA’s investigators were aware of the IIHS research and chose not to factor it into its Autosteer safety analysis out of a desire to go easy on Tesla for political or other reasons. Whether this oversight can be explained by the sheer incompetence by not one but the multiple NHTSA officials who put their name on the agency’s report or a conscious effort by a regulator to help a politically favored company market the safety of a semi-autonomous system, the situation is a heaping serving of not good.
Daily Kanban has brought this issue to the attention of sources at the Department of Transportation, and will follow up on this story when we hear back from them.
