Discovering the right metrics for scalability testing

The boom of the dot.com era is rapidly becoming a distant memory. But there’s one thing about the web economy that hasn’t changed – things can and do change fast in online industry. Rapid expansion is something that all savvy business owners should plan for. Having the right infrastructure and software management tools in place is vital if you want to take advantage of new opportunities. You never know when an ad or idea might go viral. Once your brand hits the big time, you could see a massive increase in visits on your company website over a period of just 24 hours. But you’re not going to capture and retain this influx of customers if your website suffers from poor performance as a result of system overload.

Put yourself in the consumer’s shoes for a moment

You hear the buzz about a new product or service that sounds like a great deal. Your friends are touting it on all the social media sites. The company in question has created a really funny video ad that has everyone talking. There’s a new customer special that’s good for a limited time only. You are more than intrigued – you’re anxious to check this out right away. However, when you try to visit the company’s website, it takes forever to load (just a few seconds seems like forever in today’s high-speed internet environment). Or, there’s some kind of glitch that keeps you from accessing the site at all. Instead, you get an error message. Even worse, maybe the online order placement appears to be working but the shopping cart function crashes when you try to check out. 

What should have been a breeze has now turned into a time waster. You won’t be back to visit this site again! Now you feel irritated and start posting your own opinions about the company on your FaceBook page or your Twitter feed. All those thousands of other disappointed would-be customers join you and the bad publicity soon outweighs the initial spate of good reviews. It’s a huge wasted opportunity for a company that just wasn’t prepared for success. 

Don’t just buy it – try It!

If you don’t want to be the company starring in the cautionary tale above, you need to do more than provision for scalability. Purchasing memory space or processing power for rapid scalability in the cloud is not enough. These additional resources will do you no good at all if your applications are not designed to actually function when they are hit with high demand. Doing load testing early and often in the development of a business application is the best way to ensure that it will be scalable. The next best thing is to uncover scalability issues through rigorous testing after the application is already built and deployed. The worst outcome is to find out that your application does not scale well when it’s actually put to the test in a real world situation. By that time, it’s too late. 

Fortunately, the cloud is not just the place to go for added computing power and storage during a spike in usage. It’s also the place to turn for the resources you need to perform scalability testing. You can temporarily increase your cloud usage to create the required test environment, then reduce your usage again until the time comes when you actually need to deal with a sudden spike in traffic. In addition, web-based testing outside your firewall will be more realistic than internal testing. So, as you budget for more cloud resources, don’t forget to factor in the cost of scalability testing. 

What’s involved in scalability testing?

You’ll start with gathering available data about how visitors currently use your website. This includes:

• How many pages they visit and how long they spend on each one
• Which pages are visited most often
• How the typical new or returning visitor navigates through the website
• What actions they take (filling out contact info, making a purchase, etc.)
• How many visitors are on your website at one time during your busiest periods
• How long these peak periods last
• How fast usage ramps up from low to peak

This is the type of basic information used for standard load testing of various scenarios. However, scalability testing goes further. You don’t want to just figure out how well your servers, databases, and applications handle normal peak levels. You need to see how they will perform when your company suddenly becomes enormously popular. 

Bottlenecks that might not appear under standard load testing could lead to complete failure with a sudden, extreme ramp up to sustained high levels of traffic. Scalability testing acts as a blend of load and performance testing to ensure that applications work as desired when deployed on a much larger system. You don’t just test with a QA process that confirms “good enough” performance. You test an application to the point of failure. That’s the only way to identify the bottlenecks in the system. These bottlenecks might occur in any number of places including:

• Web, application, or database servers
• Drivers
• Operating system
• The application code itself
• The firewall
• The network infrastructure

Different aspects of your system or application might also fail at different points. In addition, having more users might actually create new areas where failure can occur. For example, if you have 10,000 new visitors, it’s not just concurrent access to an application that could be a problem. You might also experience issues with the database where you need to store all the data on these new customers. Fortunately, there are many consulting firms that specialize in this type of performance testing. If you don’t have the IT resources in-house to create the right scripts for scalability testing, you may wish to outsource the process.