Load Balancers

As we scale systems, it's essential to realize the impact of all the components in our systems and how they interact. For example, load balancers usually come into play once we scale beyond one server being able to serve requests reliably.

Load Balancers

As we scale systems, it's essential to realize the impact of all the components in our systems and how they interact. For example, load balancers usually come into play once we scale beyond one server being able to serve requests reliably.

That being said, here are a few ways load balances are introduced into our systems and a few benefits they provide when scaling systems.

We will briefly discuss these concepts and what you should be aware of.

What is a load balancer?

So let's think about a traditional web server and its role in serving traffic. It receives one request, and it serves. It receives ten requests and serves them; what would happen if we had to serve 100,000 requests? Could one single server handle it? For simple static websites, no problem but for more complex workloads, probably not without a Beefy Server™.

Here is where the load balancer comes into play; it solves this problem by routing traffic across many servers so the load balancer can balance the load between all the web servers. These servers can often be thought of as a server pool.

If you were running just one server, your site was effectively down if that server crashed or was unavailable. So you want at least two servers in your pool to double your capacity effectively. Unfortunately, a load balancer with just one downstream server serves little purpose in serving more requests or improving reliability.

Load balancers are also responsible for how that load is distributed across the pool.

1. Round Robin - Requests are distributed one at a time to each server.

2. IP Hashing - The IP address of the client is hashed and pinned to a specific server which handles the request. Useful in environments where session persistence or pinning to specific servers is essential.

3. Least load - Requests are distributed to the downstream server with the least concurrent connections.

There are several benefits to introducing load balancers into your system but not limited to the following:

1. Reduced downtime

2. Scalability

3. Redundancy

4. Flexibility

5. Efficiency

What are Layers?

OSI layers broken down

The Open Systems Interconnection (OSI) model describes seven layers computer systems use to communicate over a network. It was the first standard adopted initially in the 1980s.

Load balancers (LB) are divided into two groups Layer 4 or Layer 7. Layer 4 LBs can only act on data found in the transport and network layers like IP and TCP. While Layer 7 LBs can distribute requests based on data found in the application layer, like HTTP, which can allow you to do fancy things like siphon traffic to your new service. This mechanism has come in handy a few times when using the Strangler Pattern to migrate traffic from an old system to a new one.

Strangler Architecture Pattern

An "old" system is concealed beneath a middle-man facade in the Strangler pattern. Then, behind the façade, external replacement services for the old system are gradually implemented.
The facade stands in for the system's functional entrances. 

Calls are routed through the facade to the outdated system. The old system's services are transformed into a new set of services invisibly. When a new service goes into operation, the intermediary facade is changed to divert calls that were made to the old service to the new service. Over time, the old system's services are "strangled" in favour of the new ones.

Failover

Certain types of load balancers are aware of server "health" and their current availability status; if they are performing less than ideally, LBs can take action.

In extreme scenarios were badly behaving servers are in our system, we can safely rotate them out of the pool with reduced impact on our upstream services and clients. Failover must take place quickly to avoid gaps in service. Overall significantly improving reliability and availability.

Load balancers are fundamental components to modern distributed systems. As well as great tool in your arsenal when upgrading your architecture on your scaling journey.