Six Steps to Accelerating Mobile App Testing

In the mobile enterprise, if you're not testing and releasing apps faster than your competitors, you're falling behind. Even the stuffiest insurance company knows by now that speed kills in mobile.

But while developer and QA teams are doing their best to fast-track their apps, a recent survey by the Perfecto Mobile research team showed that organizations still struggle with what we like to call velocity blockers, which are testing practices that get in the way of quick, iterative releases.

Whatever factors are slowing you down, we believe the remedy can be found in one, or perhaps all, of the following best practices for speeding up mobile app testing.

Implement automated testing

Companies should aim to automate 95% of test cases to reach an optimal level of coverage.

Manual testing is the top reason mobile app development is too slow. A few years ago, one of our top U.S. banking customers was taking too long to complete their test cycles because of excessive manual testing. App defects were regularly missed and made their way into production where they were exposed to the hypercritical eyes of users.

The bank automated its testing using robust scripts and our Continuous Quality lab as-a-service. Previously, the bank was getting by on 16-20 day regression testing cycles, but after automating, it saw a 90% time reduction per regression cycle. They brought weeks-long cycles down to a matter of hours. The bank was even able to calculate that it saved $40,000 per test cycle.

Companies should aim to automate 95% of test cases to reach an optimal level of coverage, and for maximum efficiency they should use an automated testing solution that works with the IDEs (Eclipse, Visual Studio, HP UFT) and test frameworks (Selenium, Calabash, Appium) they're already using.

Stop managing your own device lab

It costs an estimated $6,500 per device per year to manage devices in-house. That's a lot. And things becomes unmanageable – and unprofitable – as the number of devices you need to test on keeps growing.

Manual in-house testing can really set you back when new devices hit the market. Was your app ready to run on the Samsung Galaxy S6 when it released? Better yet, did you get it before it was released to test your app against it before users could upgrade? The answer is probably no. But having the right devices is vital to having the best test coverage. So how do you get the latest devices into your test lab, online and ready for testing before your users get their hands on them?

Let a managed lab-as-a-service do the work for you, where a vendor acquires new devices before they're publicly available. Let them manage your test lab for you by providing real devices, networks, locations and user conditions to test against. Make sure they also offer an SLA (service level agreement) and dedicated experts to help you.

The payoff will be that you'll have remote access to devices before they launch (no more pre-orders and store pick-ups on launch day) and you'll reap the cost savings of spending much less than the $6,500 per device.

Put your device test lab in the cloud

If your team does decide to purchase devices and use an in-house lab, they'll inevitably confront the need to make devices available to testers around the world in real time. That's tough to do with a bunch of test devices piled up on a desk.

One of our customers, a top U.S. mobile carrier, had staff members who had to acquire new-to-market devices and ship them to where testing was performed. When timelines weren't being met, the staff member physically traveled with devices to make sure they made it to the right testing group on time. Travel costs and limited testing time made for an inefficient process.

Automating your test scripts may speed up your testing and releases, but it's not enough.

Once the carrier chose to put devices in the cloud, the device costs and deadline challenges were greatly reduced. They can now centrally manage real devices accessed from a scalable, on-demand cloud. Groups can access devices remotely from anywhere while still ensuring the same user experience they'd test against if the device was in their hands.

Automate provisioning of your environment

Automating your test scripts may speed up your testing and releases, but it's not enough. You must first make sure your back-end service APIs are stable enough to support the automated tests. When you run automated tests across an unstable environment you'll get false negative alerts in your bug reports. Spending time sifting through false negatives will significantly slow down release cycles.

It's up to you to automate the provisioning of your test environment to make sure it's ready for test automation. Do so by adding pre-test sections that confirm you're environment is test-ready and can tell the difference between errors due to an unstable environment versus genuine test case failures. In the end, if your environment is stable, your tests won't return false negatives, saving you time and money.

Apply continuous quality

As developer teams begin using agile practices, the feedback loop between QA and developers tends to become unbalanced because the process is so new. A travel site customer of ours was doing monthly release cycles with three weeks of development and the last week reserved for testing and fixing bugs. The problem was that as much work needed to happen in the final week as in the previous three weeks.

The company was able to embed quality throughout its entire development lifecycle, a concept we call continuous quality. Unit tests make QA feedback always visible and developers always know where their code stood because it was constantly being tested. Maintaining this kind of control over the quality of its mobile app throughout all stages of its SDLC boosted the speed of development for the travel site and reduced the number of defects escaping into production, ultimately leading to higher user satisfaction.

Make evidence the key

It's a common scenario: a developer can't replicate the issue reported by the QA team. Is something unique to the device impacting performance? Is my network more secure, and therefore not returning a bug? Am I following all of the same steps the QA team took to find the bug?

Having evidence of defects is critical. If it's hard to reproduce bugs in testing due to lack of evidence, it will be hard to provide sufficient feedback to developer teams. With a tool that records transactions and then lets you share the video recording with team members during feedback cycles, bug reproduction won't be a problem anymore and you'll speed up the testing process.

What tips do you have for simplifying the application testing process? Let us know.