Demystifying Load Balancers: A Guide to Building Your Own

Load balancers play a pivotal role in ensuring smooth and efficient traffic distribution across servers, yet their inner workings often remain shrouded in mystery. In this comprehensive guide, we’ll unravel the intricacies of load balancing and walk you through the process of building your load balancer using Node.js.

Photo by Shane Aldendorff on Unsplash

Understanding Load Balancers

At its core, a load balancer is like a traffic cop for your servers. Its primary job is to evenly distribute incoming requests among multiple servers, ensuring that no single server gets overwhelmed while others remain idle. Think of it as a facilitator that ensures smooth and efficient communication between clients and servers. A simple load balancer is nothing more than a simple server running on a machine like your normal node server or Python’s Flask server.

Types of Load Balancers

Before delving into the nitty-gritty of implementation, let’s clarify the distinction between Layer 4 and Layer 7 load balancers. While Layer 4 load balancers operate at the transport layer and act as transparent conduits for data transfer, Layer 7 load balancers delve deeper into application-layer protocols, enabling advanced features such as content-based routing and SSL termination. For this guide, we’ll focus on crafting a Layer 4 load balancer.

The Role of Sockets

To understand how a load balancer operates, it’s essential to grasp the concept of sockets. In simple terms, a socket is a communication endpoint that allows processes to communicate with each other, either on the same machine or across a network.

Think of a socket as a two-way communication channel between a client and a server. When a client sends a request to a server, it creates a socket connection, enabling data to be transmitted back and forth.

Sockets can also facilitate communication through readable and writable streams. A readable stream represents a source from which data can be read, while a writable stream represents a destination to which data can be written.

Objective

If you think from a no-clue perspective of how a load balancer works, you can very easily create an intuition of What to do. Just take a pause and think before reading ahead.

Your main objective is to redirect a request from one machine to another(can be the same as well) such that each server receives a similar number of requests and no server is overwhelmed. But, it’s not that simple because that’s not how the network works. You can’t store/flow a “connection” between two machines. You have to create a connection with the client and the server and then find a way to help them talk to each other. After that, you would need to copy what comes in from the client and send it to the server and vice-versa.

So you copy plain bytes(forget HTTP, and HTTPS that come at the application layer) and transfer them from server to client via LB. We’ll implement a load balancer in this article using Nodejs. You can use any language and find the similar functionality that we’ll use here.

Introducing the Players:

1. lb.js: Our load balancer, the brains behind the operation.
2. server.js: The servers, are ready to handle incoming requests.
3. client.js: This will act as a client. This will make calls to the load balancer.

Let’s start with client.js:

const http = require('http');

// Our client sends a POST request to the load balancer every second
setInterval(() => {
    const options = {
        hostname: 'localhost',
        port: 3000,
        path: '/',
        method: 'POST',
    };

    // Sending a request to the load balancer
    const req = http.request(options, (res) => {
        let data = '';
        // Listening for data from the server
        res.on('data', (chunk) => {
            data += chunk;
        });
        // When response is complete, log the data received
        res.on('end', () => {
            console.log(`Received from server: ${data}`);
        });
    });

    // Writing some sample data to send to the server
    req.write('Hello');

    // Handling errors in case of connection issues
    req.on('error', (error) => {
        console.error(`Error connecting to server: ${error.message}`);
    });

    // Ending the request
    req.end();
}, 1000);

In this script, we’ve set up a client that sends a POST request to our load balancer every second. It’s a straightforward setup where the client doesn’t need to know the specifics of the server it’s hitting — this is the beauty of using a load balancer, acting as a reverse proxy abstracting the complexities behind the scenes.

We’re also using a native Node.js http module, but you could easily swap it out for other libraries like Axios for making HTTP requests.

Let’s see server.js now:

const express = require('express');
const bodyParser = require('body-parser');

const app = express();
const port = process.argv[2];

// Parsing incoming request bodies
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());

// Defining a route to handle POST requests
app.post('/', (req, res) => {
    // Logging the receipt of a POST request and its body
    console.log('Received a POST request:', req.body);

    // Sending a response containing the port on which the server is running
    res.json({ "data": port });
});



// Starting the server and listening on the specified port
app.listen(port, () => {
    console.log(`Server started on port ${port}`);
});

In this code snippet, we’re using Express.js to create a server that listens for incoming requests on a specified port. We’ve set up a route to handle POST requests, where we log the request body and respond with the port on which the server is running. This way, the client can identify which server fulfilled its request.

By running node server.js 8000, for example, we start the server on the port 8000, ready to handle incoming traffic.

Now let’s see our load balancer:

const net = require('node:net');

const backendServers = [
    { host: 'localhost', port: '8000' },
    { host: 'localhost', port: '8001' },
    { host: 'localhost', port: '8002' },
    { host: 'localhost', port: '8003' },
];

function getServerStrategy() {
    let currentServer = 0;
    return () => {
        currentServer++;
        return backendServers[currentServer % backendServers.length];
    };
}
const serverSelector = getServerStrategy();
const server = net.createServer((socket) => {
    makeConnection(serverSelector(), socket);
});

function makeConnection({ port, host }, socket) {
    const client = new net.Socket();
    
    client.connect(port, host, () => {
        console.log(`Connected to server at ${port}`);  
        socket.pipe(client); // stream data from server to client
        client.pipe(socket); // stream data from client to server
    });
    
    client.on('close', () => {
        console.log('Connection closed');
    });

    client.on('error', (error) => {
        console.error(`Error connecting to server at ${port}: ${error.message}`);
        socket.destroy();
    });

    socket.on('error', (error) => {
        console.error(`Socket error: ${error.message}`);
        client.destroy();
    });
}

const PORT = 3000;
server.listen(PORT, () => {
    console.log(`listening on ${PORT}`);
});

The Code Breakdown

In lb.js, we begin by importing the net module, which provides networking functionality. We define an array backendServers containing the details of our backend servers (host and port numbers).

The getServerStrategy function implements a round-robin server selection strategy, ensuring that incoming requests are evenly distributed among the backend servers. You can define any strategy you want. We’ll see consistent hashing in another article.

We then create a TCP server using net.createServer(), which listens for incoming client connections. Upon connection, the makeConnection function is called to establish a connection with a selected backend server.

Inside makeConnection, a new socket client is created to connect to the backend server specified by the round-robin strategy. Once the connection is established, data is streamed bidirectionally between the client and the backend server using the pipe method. pipe method streams any byte that comes from the client to the server and vice-versa. At last, we added minimal error handling as well.

Checkout the code here with the running guide.

Building Resilience

While our load balancer is pretty slick, it’s always good to plan for the unexpected. We can add features like error handling and failover mechanisms to ensure our system stays up and running, even when things get hectic.

Scaling Up

As our website grows, we might need to scale up our load balancer to handle more traffic. By adding more servers and fine-tuning our setup, we can keep things running smoothly, no matter how popular our site becomes.

Conclusion

Building a load balancer from scratch might seem daunting, but with the understanding, it’s doable in any language. So whether you’re running a small blog or managing a bustling e-commerce site, a load balancer is your secret weapon for keeping things running smoothly. Ready to take your traffic management skills to the next level? Let’s get coding!

I would highly recommend implementing your own. It’s really simple :D