Quality and consistency key to self-driving car technology
How do you test self-driving car technology when the consequences of an error can be fatal? Here are some best practices for testing automotive apps and self-driving car software.
Testing software in a traditional environment of desktop computers used to be simple. Then, came the cloud. Then, mobile. With each new emerging technology, testing becomes more and more complex. IoT, augmented and virtual reality, and AI cognitive systems all pose unique challenges.
Automotive apps and self-driving car technology in particular, though, pose a fresh set of obstacles for QA and testing initiatives. According to Brian Bernknopf, managing director at QA Consultants, adequately vetting self-driving autonomous vehicle technology will require a blend of standard processes and techniques with tools that have not yet been created.
Joint testing of autonomous vehicle software
Bernknopf's team is working with public and private international organizations to pull together the right testing environment. Their goal is to establish an environment that ensures the interoperability of connected vehicles. With so many moving parts, there's a lot to consider.
"Every OEM and several stub vendors are working on their own self-driving car technology, and then, you've got companies like Google and Uber also working on this tech," Bernknopf said. "How are they all going to work together, and how do you ensure or validate that they will work together under all circumstances?"
Right now, the search is on for answers to these questions -- and more. How do you make sure all these autonomous technologies will work with connected infrastructure and public safety requirements? How do we test all of the above without building new hardware or a whole new car every time? The stakes here are high.
The connected car as a platform
It's not just whether computer vision can see an anomaly or a potential hazard while driving, but it also involves the interoperability of the car as a platform.
Brian BernknopfManaging director, QA Consultants
Crashing a computer or a network was once the worst outcome of a software error. But today, if developers fail to consider the functionality and integration of their self-driving car vehicle technology, they'll be responsible for an actual car crash.
"It's not just whether computer vision can see an anomaly or a potential hazard while driving, but it also involves the interoperability of the car as a platform," Bernknopf said.
He further explained how, if you're developing automotive apps, for the QNX platform perhaps, you have to develop software that fits not only within those assigned vehicles, but software that works with the latest model of Toyota, Ford or any other car. And finally, you have to ensure the self-driving car technology doesn't introduce a security risk to the vehicle and thus the safety of the owner and general public.
These are all things developers have to consider now. And that's just within one vehicle, not taking into account vehicle-to-vehicle or vehicle-to-infrastructure communication. Bernknopf pointed to Tesla as the company that is leading the pack in this area by creating its own network. However, integrating across manufacturers will prove more difficult.
"For now, we're at such an early stage for autonomous vehicles that, as we continue to work with our customers and partners, each application is kind of a black box. … The idea of taking an autonomous vehicle and putting it into a software simulation or lab for testing is very complex."
However, in Bernknopf's opinion, the cost-benefit analysis clearly shows that testing these software components in a lab environment is essential. And fortunately, each partner is stepping up to play a role in the testing process based on capabilities and strengths.
"The manufacturers and OEMs are doing a lot of investigation into the usability testing of these platforms," Bernknopf said. "There're a lot of simple things that end up being complex, like the ergonomics of interfacing with a UI while in the vehicle. That's typically part of the R&D at the manufacturers."
Best practices for testing autonomous vehicles
For Bernknopf and his team, testing automotive apps and driverless car technology means building novel multivariate testing systems. But the principles underlying their approach to testing and QA remain tried-and-true:
Focus on the changes that are being introduced.
Separate functionality from integration.
Automate aspects of testing in the future by developing the appropriate APIs.
Add tooling that is developed as the technology matures.
Above all else, Bernknopf stressed the importance of taking a risk-based QA approach. With so much riding on having well-designed and thoroughly tested software driving the autonomous vehicle market, that's a wise decision.
Dig Deeper on Software development best practices and processes
