Load Balancing

Overview

Load balancing distributes incoming traffic across multiple healthy backend instances to improve availability and responsiveness.

Load Balancing Algorithms

• Round robin
• Least connections
• Weighted round robin
• Consistent hashing

flowchart LR
    C[Client Requests] --> LB[Load Balancer]
    LB --> A[App Instance A]
    LB --> B[App Instance B]
    LB --> D[App Instance C]

Production Example

A global API usually combines DNS-based routing with regional load balancers.

upstream api_pool {
    least_conn;
    server api-1.internal:8080;
    server api-2.internal:8080;
    server api-3.internal:8080;
}

server {
    listen 80;
    location / {
        proxy_pass http://api_pool;
    }
}

Health and Resilience

• Active health probes remove unhealthy targets
• Connection draining supports zero-downtime rollouts
• Rate limiting protects backends during bursts

Common Pitfalls

• Missing session strategy for stateful workloads
• Using defaults without validating algorithm fit
• Insufficient timeout tuning for slow downstream services

Key Takeaways

• Load balancing is a core reliability and scale component.
• Health checks and draining are essential for safe deployments.
• Algorithm choice should align with traffic patterns.