Load Balancers

As we scale systems, it's essential to realize the impact of all the components in our systems and how they interact. For example, load balancers usually come into play once we scale beyond one server being able to serve requests reliably.

That being said, here are a few ways load balances are introduced into our systems and a few benefits they provide when scaling systems.

We will briefly discuss these concepts and what you should be aware of.

What is a load balancer?

So let's think about a traditional web server and its role in serving traffic. It receives one request, and it serves. It receives ten requests and serves them; what would happen if we had to serve 100,000 requests? Could one single server handle it? For simple static websites, no problem but for more complex workloads, probably not without a Beefy Server™.

Here is where the load balancer comes into play; it solves this problem by routing traffic across many servers so the load balancer can balance the load between all the web servers. These servers can often be thought of as a server pool.

If you were running just one server, your site was effectively down if that server crashed or was unavailable. So you want at least two servers in your pool to double your capacity effectively. Unfortunately, a load balancer with just one downstream server serves little purpose in serving more requests or improving reliability.

Load balancers are also responsible for how that load is distributed across the pool.

1. Round Robin - Requests are distributed one at a time to each server.
2. IP Hashing - The IP address of the client is hashed and pinned to a specific server which handles the request. Useful in environments where session persistence or pinning to specific servers is essential.
3. Least load - Requests are distributed to the downstream server with the least concurrent connections.