Architecture Notes — System Design &  Software Development

Architecture Notes — System Design &  Software Development

Share this post

Architecture Notes — System Design &  Software Development
Architecture Notes — System Design & Software Development
Load Balancers
Copy link
Facebook
Email
Notes
More
User's avatar
Discover more from Architecture Notes — System Design & Software Development
In your inbox every Sunday! ⚡️
Over 28,000 subscribers
Already have an account? Sign in

Load Balancers

As we scale systems, it's essential to realize the impact of all the components in our systems and how they interact. For example, load balancers usually come into play once we scale beyond one server being able to serve requests reliably.

Mahdi Yusuf's avatar
Mahdi Yusuf
Nov 02, 2022
1

Share this post

Architecture Notes — System Design &  Software Development
Architecture Notes — System Design & Software Development
Load Balancers
Copy link
Facebook
Email
Notes
More
Share
Load Balancers
Load Balancers

As we scale systems, it's essential to realize the impact of all the components in our systems and how they interact. For example, load balancers usually come into play once we scale beyond one server being able to serve requests reliably.

That being said, here are a few ways load balances are introduced into our systems and a few benefits they provide when scaling systems.

We will briefly discuss these concepts and what you should be aware of.

What is a load balancer?

So let's think about a traditional web server and its role in serving traffic. It receives one request, and it serves. It receives ten requests and serves them; what would happen if we had to serve 100,000 requests? Could one single server handle it? For simple static websites, no problem but for more complex workloads, probably not without a Beefy Server™.

Here is where the load balancer comes into play; it solves this problem by routing traffic across many servers so the load balancer can balance the load between all the web servers. These servers can often be thought of as a server pool.


If you were running just one server, your site was effectively down if that server crashed or was unavailable. So you want at least two servers in your pool to double your capacity effectively. Unfortunately, a load balancer with just one downstream server serves little purpose in serving more requests or improving reliability.

Load balancers are also responsible for how that load is distributed across the pool.

  1. Round Robin - Requests are distributed one at a time to each server.

  2. IP Hashing - The IP address of the client is hashed and pinned to a specific server which handles the request. Useful in environments where session persistence or pinning to specific servers is essential.

  3. Least load - Requests are distributed to the downstream server with the least concurrent connections.

There are several benefits to introducing load balancers into your system but not limited to the following:

  1. Reduced downtime

  2. Scalability

  3. Redundancy

  4. Flexibility

  5. Efficiency

What are Layers?

OSI layers broken down
OSI layers broken down

The Open Systems Interconnection (OSI) model describes seven layers computer systems use to communicate over a network. It was the first standard adopted initially in the 1980s.

Load balancers (LB) are divided into two groups Layer 4 or Layer 7. Layer 4 LBs can only act on data found in the transport and network layers like IP and TCP. While Layer 7 LBs can distribute requests based on data found in the application layer, like HTTP, which can allow you to do fancy things like siphon traffic to your new service. This mechanism has come in handy a few times when using the Strangler Pattern to migrate traffic from an old system to a new one.

Strangler Architecture Pattern

An "old" system is concealed beneath a middle-man facade in the Strangler pattern. Then, behind the façade, external replacement services for the old system are gradually implemented.
The facade stands in for the system's functional entrances. 

Calls are routed through the facade to the outdated system. The old system's services are transformed into a new set of services invisibly. When a new service goes into operation, the intermediary facade is changed to divert calls that were made to the old service to the new service. Over time, the old system's services are "strangled" in favour of the new ones.

Failover

Certain types of load balancers are aware of server "health" and their current availability status; if they are performing less than ideally, LBs can take action.

In extreme scenarios were badly behaving servers are in our system, we can safely rotate them out of the pool with reduced impact on our upstream services and clients. Failover must take place quickly to avoid gaps in service. Overall significantly improving reliability and availability.

Load balancers are fundamental components to modern distributed systems. As well as great tool in your arsenal when upgrading your architecture on your scaling journey.


Subscribe to Architecture Notes — System Design & Software Development

By Mahdi Yusuf · Hundreds of paid subscribers
In your inbox every Sunday! ⚡️
Pablo Tapia's avatar
1 Like
1

Share this post

Architecture Notes — System Design &  Software Development
Architecture Notes — System Design & Software Development
Load Balancers
Copy link
Facebook
Email
Notes
More
Share

Discussion about this post

User's avatar
Redis Explained
A deep technical dive into all things Redis. Covering various Redis topologies, data persistence and process forking.
Aug 11, 2022 • 
Mahdi Yusuf
154

Share this post

Architecture Notes — System Design &  Software Development
Architecture Notes — System Design & Software Development
Redis Explained
Copy link
Facebook
Email
Notes
More
6
Architecture Notes is Now on Substack!
Learn what's changing (and what's not) as we join the Substack Network!
May 21, 2024 • 
Mahdi Yusuf
70

Share this post

Architecture Notes — System Design &  Software Development
Architecture Notes — System Design & Software Development
Architecture Notes is Now on Substack!
Copy link
Facebook
Email
Notes
More
Database Sharding Explained
More features, active users, and data are collected daily. Your database is slowing your application. Many people don't understand database sharding…
Dec 12, 2022 • 
Mahdi Yusuf
73

Share this post

Architecture Notes — System Design &  Software Development
Architecture Notes — System Design & Software Development
Database Sharding Explained
Copy link
Facebook
Email
Notes
More
1

Ready for more?

© 2025 Mahdi Yusuf
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More

Create your profile

User's avatar

Only paid subscribers can comment on this post

Already a paid subscriber? Sign in

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.