Automate Web server load balancing for high scalability

What is load balancing? How does it work?

Load balancing is the process of routing inbound Web traffic to multiple servers to help ensure that resources are used effectively and that no one server takes on too much work at once. Distributing the demand among a network of servers can limit delays and speed response time. With load balancing, if one or more servers go offline due to failure or for routine maintenance, an application will not necessarily shut down completely. Instead, the remaining servers simply pick up the slack. You can have multiple servers on-site, or you can choose to use distributed servers in a cloud networking model.

In a distributed computing cloud, servers may be located in secure data centers in many different geographic regions. Even if one data center got hit by a natural disaster, there would still be plenty of servers available in other locations to handle the workload. With some private clouds that boast a full Infrastructure as a Service (IaaS) package, load balancing occurs among a set of dedicated servers that are reserved for one particular enterprise customer.

In the public cloud, your IP traffic load is balanced along with that of many other clients on a broadly distributed network of servers. From a true scalability perspective, load balancing is probably simplest in the public cloud where server resources from major cloud providers are as close to unlimited as you can get. Of course, it's important to check the track record of any potential cloud provider to see if their customers really do experience effortless scalability on demand.

A focus on scalability doesn’t mean that the entire process has to be shifted to the cloud. There are hybrid options available. For example, Amazon offers two EC2 versions of Zeus (Traffic Manager and Simple Load Balancer) that allow enterprise and small business customers to balance Web traffic loads in the cloud on an as-needed basis. This gives customers the option to provision on-site for normal traffic and then scale up into the cloud temporarily during times of peak traffic.

Talking to vendors about load balancing

Here are some of the features to ask about when shopping for load balancing tools that are scalable:

• The ability to collect detailed real-time statistics to use in distributing the workload. Using historical data alone to make predictions is a sure-fire way to end up guessing wrong.
• A virtualized network environment that presents resources as a single-server device. This is an important feature when it comes to administrating cloud applications. A simple interface makes management tasks easier.
• A load balancing service that can “see” the load at the application layer and not just at the instance layer. This is a factor in scalability in the cloud where there may be a huge number of application instances running simultaneously.
• The ability to create new instances of the load balancer itself and configure balancers to meet custom-requirements if necessary. Ideally, there should be an Application Programming Interface (API) that supports a high level of integration for your specific business applications.
• Load balancing that includes monitoring functions to identify unhealthy application instances and redirect traffic to healthy instances. Just because a server or application is “available” doesn’t mean it’s functioning optimally.
• The ability to automate scaling so that more additional virtual servers are added based on preset thresholds – and automatic rebalancing of the workload between the existing and new servers whenever this occurs. This rebalancing should occur with no service interruption, rebooting, or other actions required on the part of IT.

Automation matters

Load balancing in the cloud sounds great in theory. But it doesn’t just happen by magic. For example, an enterprise organization might choose to pursue storage consolidation on a Storage Area Network (SAN) to better utilize server hardware resources. There are many companies that have tried this approach and found that they are constantly re-spreading workloads to avoid space or storage restrictions.

This is a labor intensive, recurring task in which data volume tends to increase over time and workloads are inherently disposed toward fluctuation. Handling this process manually involves analyzing current and predicted capacity and performance needs, provisioning additional storage, reconfiguring applications, shutting down and restarting servers and applications, etc. This process must be repeated every time there’s a significant change in workload or some servers will be overloaded while others remain idle.

In such a scenario, there will always be resources that are paid for but no used. It's also possible for load balancing tasks to use up all of an IT administrator’s available hours. Even worse, because the types of predictions and calculations required to perform proper load balancing are so complex and interconnected, mistakes will occasionally occur. Any miscalculation adds to server downtime when the need to scale up or scale back arises.

Virtualization that eliminates the need to manually reconfigure resources and automation that ensures data is optimally placed in the SAN save the IT department time and money. These features can greatly improve performance. It’s simply a matter of software doing the job better than people. There are a number of mundane, repetitive systems-maintenance tasks that fit this category – well beyond this SAN example.

Of course, automation is only as good as the scripts that are created to carry it out. It doesn’t matter how much the hardware scales if you aren't designing your Java applications to make the best use of your servers – whether on-site or in the cloud.