Impressive and touching as this demonstration is, it is also deceptive. Google’s cars follow a route that has already been driven at least once by a human, and a driver always sits behind the wheel, or in the passenger seat, in case of mishap. This isn’t purely to reassure pedestrians and other motorists. No system can yet match a human driver’s ability to respond to the unexpected, and sudden failure could be catastrophic at high speed.

But if autonomy requires constant supervision, it can also discourage it. Back in his office, Reimer showed me a chart that illustrates the relationship between a driver’s performance and the number of things he or she is doing. Unsurprisingly, at one end of the chart, performance drops dramatically as distraction increases. At the other end, however, where there is too little to keep the driver engaged, performance drops as well. Someone who is daydreaming while the car drives itself will be unprepared to take control when necessary.

Reimer also worries that relying too much on autonomy could cause drivers’ skills to atrophy. A parallel can be found in airplanes, where increasing reliance on autopilot technology over the past few decades has been blamed for reducing pilots’ manual flying abilities. A 2011 draft report commissioned by the Federal Aviation Administration suggested that overreliance on automation may have contributed to several recent crashes involving pilot error. Reimer thinks the same could happen to drivers. “Highly automated driving will reduce the actual physical miles driven, and a driver who loses half the miles driven is not going to be the same driver afterward,” he says. “By and large we’re forgetting about an important problem: how do you connect the human brain to this technology?”

Norman argues that autonomy also needs to be more attuned to how the driver is feeling. “As machines start to take over more and more, they need to be socialized; they need to improve the way they communicate and interact,” he writes. Reimer and colleagues at MIT have shown how this might be achieved, with a system that estimates a driver’s mental workload and attentiveness by using sensors on the dashboard to measure heart rate, skin conductance, and eye movement. This setup would inform a kind of adaptive automation: the car would make more or less use of its autonomous features depending on the driver’s level of distraction or engagement.

Ref: Proceed with Caution toward the Self-Driving Car – MIT Technology Review