Autonomous Vehicle Safety Driver Fault Injection (2019)

​While working at Lyft, in the autonomous vehicle division, I was responsible for designing and executing a rigorous safety driver safety testing protocol. Collaborating with an engineer, Sasanka Nagavalli, I designed a program of various injected faults to test different aspects of safety driver alertness and reaction times. The program was reversible to counter order effects and contained both errors of omission and errors of commission.

• Errors of omission are when the vehicle fails to do something it should, such as failing to stop at a stop sign or failure to execute a planned turn or to stay in lane; these more subtle faults tested driver alertness as well as reaction times.
• Errors of commission are when the vehicle makes an unexpected positive action such as a sudden acceleration, braking, or swerving; these much more obvious faults mainly tested driver reaction times.

Safety drivers were accompanied by an instructor passenger who would tell them what the next expected move of the vehicle would be as it drove around a testing ground with road markings, stop signs and traffic lights. The safety drivers were told to correct the AV's behavior if anything unexpected happened. Of course it did, since programmed faults were triggered when the vehicle arrived at a preprogrammed geolocation. 

We also ran the same program with non-safety drivers to show that the Lyft safety drivers performed significantly better, which demonstrated that the training they were given was improving their driving skills.

In all tests the instructor in the passenger seat knew what was coming ahead of time and was ready to assist the driver if they did not steer correctly in time. And all tests were run at a fairly low speed to make sure the protocol itself could be conducted safely.