In server-side load balancing, a central load balancer distributes incoming traffic across multiple backend servers to ensure that no single server is overwhelmed with requests.
NGINX
One of the most widely used tools for server-side load balancing is NGINX. Unlike client-side load balancing (where the client makes decisions about which server to use), server-side load balancing occurs on a dedicated machine that sits between the client and the backend services.
What is NGINX?
NGINX is a powerful open-source web server that also functions as a reverse proxy, load balancer, and HTTP cache. It is commonly used to manage incoming HTTP, TCP, and UDP traffic and distribute it efficiently across multiple backend servers.
How Server-Side Load Balancing Works with NGINX?
When a client sends a request to a system using server-side load balancing, the request is first received by the load balancer (NGINX in this case). NGINX then selects a backend server (from a pool of servers) based on a predefined load-balancing algorithm and forwards the request to that server. Once the backend server processes the request, it sends the response back to NGINX, which in turn forwards it to the client.
NGINX Load Balancing Architecture
Client: Sends requests to the load balancer (NGINX).
NGINX (Load Balancer): Distributes incoming requests to one of the backend servers based on a load-balancing algorithm.
Backend Servers: A pool of servers (e.g., API servers, web servers) that handle the actual processing of requests.
Example
Let's create a sample service maven project say sample-project with an client API endpoint.
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
class SampleController {
@GetMapping("/api/hello-world")
public String sayHelloWorld() {
return "Hello world from " + System.getenv("SERVER_INSTANCE");
}
}
We may notice the use of SERVER_INSTANCE variable. We will be setting it the docker compose file as a environment variable since we will need to create multiple instance (more than 1) of this service to test the load balancing feature.
Create a main application file
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
application.yaml file
server:
port: 8080
Build the project to generate jar file (to be used later in docker compose file)
We will use the generated jar file directly in docker-compose instead of creating docker image for above service
Now, we need to setup NGINX.
Create a nginx.conf config file
# Define a load-balancing group named 'backend'
upstream backend {
# Backend server 1, running a Spring Boot app instance on port 8080
server backend1:8080;
# Backend server 2, running another Spring Boot app instance on port 8080
server backend2:8080;
}
server {
# Listen for incoming HTTP requests on port 90
listen 90;
# Define the default location block for handling requests specifically for /api path
location /api/ {
# Proxy pass - forwards the client requests to the 'backend' upstream group
proxy_pass http://backend;
# Preserve the original Host header from the client request
proxy_set_header Host $host;
# Pass the real IP address of the client to the backend server
proxy_set_header X-Real-IP $remote_addr;
# Include any prior X-Forwarded-For headers and append the client's IP address
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Specify the protocol (HTTP or HTTPS) used by the client
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Create a docker file for nginx with above config file to create a nginx image
Dockerfile
# Use the official NGINX base image
FROM nginx:latest
# Remove the default configuration
RUN rm /etc/nginx/conf.d/default.conf
# Copy the custom NGINX configuration file
COPY nginx.conf /etc/nginx/conf.d/
# Expose the port that NGINX will run on
EXPOSE 90
Note that config file is kept at the same place as Dockerfile and relative path is being used
Now, let us create docker image using above docker file
docker build -t nginx-load-balancer .
Now, we have the services ready. Let's create docker-compose.yml file
We can keep this docker-compose file in root of sample-project since we need to provide relative path of jar file
version: '3'
services:
nginx:
image: nginx-load-balancer
ports:
- "90:90"
depends_on:
- backend1
- backend2
backend1:
image: openjdk:17-jdk-slim
environment:
- SERVER_INSTANCE=Backend1
volumes:
- ./target/sample-project-1.0-SNAPSHOT.jar:/app/sample-project-1.0-SNAPSHOT.jar # Attach the pre-built JAR file
working_dir: /app
command: ["java", "-jar", "sample-project-1.0-SNAPSHOT.jar"] # Run the JAR file directly
ports:
- "8081:8080"
backend2:
image: openjdk:17-jdk-slim
environment:
- SERVER_INSTANCE=Backend2
volumes:
- ./target/sample-project-1.0-SNAPSHOT.jar:/app/sample-project-1.0-SNAPSHOT.jar # Attach the pre-built JAR file
working_dir: /app
command: ["java", "-jar", "sample-project-1.0-SNAPSHOT.jar"] # Run the JAR file directly
ports:
- "8082:8080"
networks:
shared-network:
driver: bridge
