Site Reliability

Configure load balancer health probes and backend redundancy

Two settings decide whether an Azure Load Balancer keeps serving when a backend instance dies: a health probe that detects the failure and stops sending traffic to it, and a backend pool with more than one instance so there is somewhere healthy left to send it.

14 min·10 sections·AZURE

Last reviewed 18 June 2026

Health probes and backend redundancy: the basics

What does a load balancer with no safety net actually look like?

An Azure Load Balancer is a layer-4 pass-through that spreads inbound TCP and UDP traffic across the instances in a backend pool. On its own it does not know whether any of those instances is actually serving. That job belongs to the health probe: a small check, TCP, HTTP or HTTPS, that the load balancer sends to each backend on a fixed interval. When a backend stops answering the probe, the load balancer marks it unhealthy and stops sending it new connections. Without a probe, or with a probe that checks the wrong thing, the load balancer keeps cheerfully forwarding requests to an instance that is hung, mid-deploy or returning errors, and your users get the failures directly.

The probe is only half the picture. A probe that correctly removes a sick instance is useless if there is nowhere else to send the traffic. If the backend pool holds a single instance and that instance goes unhealthy, every request to the pool fails, because redundancy is the thing that makes a probe actionable. This is why Azure Advisor raises the reliability recommendation 'Ensure backend pools contain at least two instances': a single-instance pool is a single point of failure no probe can rescue. Microsoft is explicit that the Standard Load Balancer SLA applies only when you have at least two healthy backend instances configured.

Most reliability gaps here are drift, not design. A load balancer stood up with one VM 'for now' and never scaled out. A probe pointed at a generic open port that answers even when the application behind it is broken. A pool where two VMs happen to live in the same availability zone, so a single datacentre fault takes both down at once. The work is to make sure every load-balancing rule has a probe that reflects real application health, and that every backend pool it feeds has at least two healthy instances spread across fault boundaries.

In this lesson you will learn how Azure Load Balancer uses health probes to detect failing backends, why a probe is only as good as the redundancy behind it, and how to find and fix the single-instance pools and weak probes in your estate. It is grounded in the Azure Advisor reliability recommendation 'Ensure backend pools contain at least two instances', and in how probes actually decide which instances receive traffic.

Fun fact

The probe that pointed at the wrong door

A classic failure mode is a health probe that checks a port rather than the application. A team points a TCP probe at port 80, the web server keeps the socket open, and the probe stays green, even while the application behind it is returning 500s or stuck waiting on a dead database. The load balancer, seeing a healthy probe, keeps routing real users straight into the failure. The fix is to make the probe test something that fails when the application fails: an HTTP probe against a dedicated health path that itself checks the application's critical dependencies. A probe is only as honest as the thing it asks.

Finding single-instance pools and weak probes

Priya runs platform reliability at a growing SaaS company. Azure Advisor has flagged several load balancers with the recommendation 'Ensure backend pools contain at least two instances', and an incident last month, a single VM rebooting for patches and taking a service down with it, made the point in person.

Rather than guess, Priya starts by listing every load balancer and counting the backend addresses in each pool, so the genuine single points of failure stand out before anything is changed.

List every load balancer and how many instances sit in each backend pool. A count of one is a single point of failure.

$ az network lb list --query "[].{lb:name, pool:backendAddressPools[0].name, instances:length(backendAddressPools[0].backendIPConfigurations)}" -o table

Lb Pool Backends

--------------- ---------- ----------

lb-checkout-prod pool-web 1

lb-api-prod pool-api 3

# lb-checkout-prod has ONE backend. A reboot there is a checkout outage. Fix first.

A pool with a single backend is the highest-value target: the most business-critical single-instance pool is where one routine failure becomes a customer-facing outage. Count instances, then check the probe behind each pool.

How a probe decides an instance is healthydeep dive

A health probe checks each backend on a fixed interval, 5 seconds by default in the Azure portal, and uses a threshold of consecutive results to flip an instance between healthy and unhealthy. A TCP probe opens a connection to the configured port and fails if the instance does not answer or sends a reset. An HTTP or HTTPS probe issues a GET against a path you specify and treats only an HTTP 200 as healthy: any other status, a 403, 404 or 500, or no response within the timeout, counts as a failure. For TCP probes the threshold is symmetric, so with a threshold of 2 an instance needs two consecutive successes to come into rotation and two consecutive failures to leave it. The probe source is always the Azure address 168.63.129.16, identified by the AzureLoadBalancer service tag, so your network security groups and host firewalls must allow it or every instance will look down.

What happens when an instance fails the probe depends on the SKU, and the difference is the whole reason redundancy matters. On a Standard Load Balancer, when a single instance probes down the load balancer simply stops sending it new connections and routes new flows to the remaining healthy backends, while established TCP flows to other instances continue. But when every instance in a pool probes down, no new flows are sent to the pool at all: a single-instance pool whose one instance fails is, by definition, an all-down pool, so the probe has done its job and there is still nothing to serve the request. On a Basic Load Balancer the all-down case is worse, because it terminates established TCP flows too.

This is why the probe and the redundancy are one capability rather than two settings. The probe is the detector; the second instance is the thing that makes detection actionable. Microsoft ties them together in the SLA itself: the Load Balancer SLA applies only when you have at least two healthy backend instances configured, and it explicitly excludes downtime caused by network security groups blocking traffic, which is exactly what a misconfigured probe path looks like to the platform.

What is the impact of a single-instance pool or a weak probe?

The direct impact is downtime. A backend pool with one instance has no redundancy: any event that takes that instance offline, a patch reboot, a failed deployment, a crashed process, an availability-zone fault, takes the whole load-balanced service offline with it. The probe will correctly detect the failure, but with nowhere healthy to route to, every request to the pool fails until the instance recovers. The failures that hurt most are the routine ones, because they are frequent and entirely foreseeable.

The second-order impact is a false sense of safety. A load balancer in front of a service looks like resilience, and a green probe looks like health, so a single-instance pool or a probe pointed at a bare port can pass every casual inspection right up until the instance it depends on fails. A probe that tests a generic open port instead of real application health is worse than no probe, because it actively keeps routing users into a broken application while reporting that everything is fine.

There is a contractual dimension too. Microsoft only offers the Load Balancer SLA when at least two healthy backend instances are configured, so a single-instance pool is outside the guarantee you are paying a Standard SKU for. If you in turn promise uptime to your own customers, a single point of failure in front of a revenue service is an SLA commitment you cannot actually back, and the credits you owe when it breaks are a real and recurring cost.

How do you make a load-balanced service survive a failure?

Work this as one loop rather than chasing individual Advisor items. The order matters: add redundancy and a real probe together, because either one alone leaves the service exposed.

1. Inventory pools by instance count and probe quality

List every load balancer, count the backends in each pool, and check what each probe actually tests. A pool with one instance is a single point of failure. A probe pointed at a bare TCP port, rather than an application health path, is a check that can stay green while the service is broken. Treat this inventory as the source of truth, not the raw Advisor count.

2. Add a second instance, spread across zones

Bring every production pool to at least two instances so the SLA applies and a single failure has somewhere to fail over to. Place those instances in different availability zones, ideally behind a virtual machine scale set, so a single datacentre fault cannot take both down at once. Two instances in the same zone is better than one, but it is not zone resilience.

3. Point the probe at real application health

Prefer an HTTP or HTTPS probe against a dedicated health endpoint that checks the application's critical dependencies, rather than a bare TCP port that answers regardless of application state. Tune the interval and threshold to detect failure quickly without flapping on transient blips, and make sure the AzureLoadBalancer source IP 168.63.129.16 is allowed through every network security group and host firewall.

4. Make redundancy the default in code

Define load balancers, pools and probes in Bicep or another infrastructure-as-code template so a redundant, properly probed configuration is the only way a service ships. A standard module with a minimum of two instances and a real health path turns reliability from something each team has to remember into something the platform enforces by default.

# 1. Add an HTTP health probe that tests a real application path (not a bare port).
az network lb probe create \
  --resource-group rg-prod \
  --lb-name lb-checkout-prod \
  --name probe-checkout-health \
  --protocol Http \
  --port 8080 \
  --path /healthz \
  --interval 5 \
  --threshold 2

# 2. Point the load-balancing rule at that probe so unhealthy instances are pulled.
az network lb rule create \
  --resource-group rg-prod \
  --lb-name lb-checkout-prod \
  --name rule-checkout-https \
  --protocol Tcp \
  --frontend-port 443 \
  --backend-port 8080 \
  --probe-name probe-checkout-health \
  --backend-pool-name pool-web

# --- Or define the whole thing in Bicep so redundancy is the default ---
#
# resource lb 'Microsoft.Network/loadBalancers@2025-03-01' = {
#   name: 'lb-checkout-prod'
#   location: location
#   sku: { name: 'Standard' }
#   properties: {
#     // Back the pool with a VM scale set of >= 2 instances across zones.
#     probes: [
#       {
#         name: 'probe-checkout-health'
#         properties: {
#           protocol: 'Http'
#           port: 8080
#           requestPath: '/healthz'
#           intervalInSeconds: 5
#           numberOfProbes: 2
#         }
#       }
#     ]
#   }
# }

Quick quiz

Question 1 of 5

Azure Advisor flags a load balancer with the recommendation 'Ensure backend pools contain at least two instances'. Why does a single-instance pool defeat the point of having a health probe?

Keep learning

Go deeper on how Azure Load Balancer probes detect failure and how to design backend redundancy that actually survives one.

You can now treat load-balancer resilience as one capability rather than a scatter of settings: inventory every pool by instance count and probe quality, add a second instance spread across availability zones so a single failure has somewhere to go, point each probe at a real application health path so a green check means a healthy service, and make that redundant shape the default in code. A health probe detects the failure; the second instance is what turns detection into a non-event.

Back to the library

Health probes and backend redundancy: the downtime view

A near-zero-cost change that converts a guaranteed outage into a non-event

This is one of the cheapest pieces of resilience you can buy in Azure. Configuring a health probe costs nothing: Azure bills a load balancer on the number of rules and the data it processes, not on its probes. The only real spend is the second backend instance that turns a single point of failure into a redundant pair, and even that is bounded and predictable: it is one more VM in a scale set, or one more small instance, not a new architecture.

Frame the decision as expected downtime, not features. A single-instance pool behind a load balancer is not 'mostly fine': it is an outage waiting for a routine event, a patch reboot, a failed deployment, a hung process, an availability-zone blip. When that instance goes down, the whole service goes down, and the probe has nowhere to fail over to. The cost of the second instance is fixed and small. The cost of the outage it prevents is variable and potentially large.

The arithmetic is one-sided. The remediation is a probe (free) plus one extra instance (a known monthly figure you can put on a line). The downside of leaving a single-instance pool is the modelled cost of an outage for that service: lost transactions, SLA credits owed to your own customers, support load and the engineering scramble of an unplanned incident. Resilience here is almost always cheaper than the first outage it would have absorbed.

This lesson is for the finance partner who sees a reliability recommendation about backend pools and wants to know what it costs to fix and what it costs to ignore. It covers why the probe itself is free, why the second instance is a small bounded line, and how to weigh that against the modelled cost of the outage a single-instance pool guarantees.

Fun fact

The probe that pointed at the wrong door

How a finance partner frames the redundancy decision

Priya is the finance partner supporting the platform team after a patch reboot took checkout offline for forty minutes during the working day. Advisor has flagged four load balancers with single-instance backend pools. The instinct is not to wave through a blanket 'double everything' budget; it is to ask what each service is worth and what one more instance costs. The probe work is free, Azure does not bill for health probes, so the only spend is the extra backend instances, and that is a small, known monthly figure per VM.

Priya asks the team to put a downtime cost against each single-instance pool. The checkout load balancer fronts revenue directly: an hour down has a measurable price in lost orders and SLA credits, so a second instance is trivially justified. An internal admin tool behind another single-instance pool can tolerate a short reboot, so it is a lower priority. The finance pack reads: 'The probe fixes are free. Two of the four single-instance pools front revenue and get a second instance now, at a fixed and tiny monthly cost against a modelled outage cost orders of magnitude larger. The other two are scheduled. We are buying out a guaranteed outage for the price of one small VM.'

Why a single-instance pool belongs on the risk register

The cost model here is asymmetric in the way reliability work usually is. Adding redundancy costs one more instance, a fixed, forecastable monthly figure, while leaving a single-instance pool in place creates an open-ended downtime liability. Each single-instance pool in front of a real service is an outage waiting for a routine event, and routine events are not rare: machines get patched, deployments fail, zones blip. The probability of at least one of them landing on an unprotected service over a year is not small.

The modelled cost of an outage on a revenue-facing service includes lost transactions during the downtime, SLA credits owed to your own customers, the support and incident-response load, and the slower erosion of trust when a service is seen to fall over. Against that, the remediation is a free probe plus one extra instance per service. The finance role is to attach a downtime cost to each single-instance pool so the work is ordered by what an outage there would actually cost, not by the order Advisor happens to list them.

What finance can do about the redundancy gap

Finance cannot add an instance, but it owns the framing that turns a list of Advisor items into a downtime-ordered plan with one free fix and one small, bounded cost. Three levers.

1. Separate the free probe work from the per-instance cost

Make explicit that the health-probe half of this is free: Azure bills a load balancer on rules and data processed, not on probes, so pointing a probe at a real health path costs nothing. The only spend is the second backend instance, a fixed, forecastable monthly figure per service. Budget the extra instances as discrete lines and treat the probe improvements as engineering time, so nobody can claim the whole capability is an open-ended cost.

2. Order the work by downtime cost, not by Advisor count

A single-instance pool fronting checkout and a single-instance pool fronting an internal admin tool both show up as the same recommendation, but an hour of downtime on each has wildly different costs. Ask the team to attach a modelled outage cost to each single-instance pool and fund the revenue-facing ones first. The metric that belongs on the reliability review is single points of failure weighted by what an outage there would cost, not the raw recommendation total.

3. Tie the second instance to the SLA you owe downstream

Where you promise uptime to your own customers, a single-instance pool in front of that service is an SLA you cannot actually back, and the credits you owe when it fails are a recurring cost. Frame the second instance as the thing that makes both Microsoft's Load Balancer SLA and your own customer commitment real. That reframes it from a discretionary spend into the cost of being able to keep a promise you have already made.

Quick quiz

Question 1 of 5

Advisor flags four single-instance load-balancer pools. As the finance partner, what is the right starting assumption about the cost to remediate?

Keep learning

Go deeper on how Azure Load Balancer probes detect failure and how to design backend redundancy that actually survives one.

You have finished the finance view of load-balancer resilience. You know the cost model is one-sided: the probe work is free, the only spend is one extra instance per service at a fixed monthly cost, and the downside of a single-instance pool is the open-ended cost of an outage on the service it fronts. Next time a backend-pool recommendation lands, you will order the fixes by modelled downtime cost, fund the revenue-facing services first, and frame the second instance as the cost of backing an SLA you have already promised.

Back to the library

Health probes and backend redundancy: the headline

Whether a single failed instance can take a whole service offline

This is the difference between an outage and a non-event. When a backend instance fails, and they always eventually do, a correctly probed, redundant load balancer notices within seconds, stops sending it traffic, and the service keeps running on the healthy instances. A single-instance pool, or a pool with no real health check, turns that same routine failure into a customer-facing outage.

The leadership question is not 'is the load balancer configured?' It is 'if one instance dies during business hours, do customers notice?' The defensible answer is no: every load-balancing rule has a probe that reflects real application health, and every pool behind it has at least two healthy instances spread across availability zones, so no single failure is a single point of failure.

This is a cost-of-resilience decision, and it is an easy one. The protection costs a probe (free) and, at most, one additional instance. The thing it prevents is an avoidable outage with a real revenue and reputation cost. Few resilience trades are this lopsided in favour of acting.

A short read for the leader who needs to know what a single-instance load-balanced service really is, a one-event-away outage, why fixing it is a small and obvious cost-of-resilience decision, and what a defensible end state looks like across the estate.

Fun fact

The probe that pointed at the wrong door

What it looks like when one dead instance stops being an outage

After checkout went down for forty minutes because a single VM rebooted for patching, the question at the next review was blunt: why did one machine restarting take down a revenue service? The honest answer was that the service had only ever run one instance behind its load balancer, so the probe correctly noticed the failure and then had nowhere healthy to send the traffic.

The team's response was to stop treating this as one incident and start treating it as a pattern. They listed every load balancer with a single-instance pool, added a second instance to the revenue-facing ones across two availability zones, pointed each probe at a real application health path instead of a bare port, and set a standard that no production load-balancing rule ships against a single-instance pool. The next time a backend instance rebooted, during business hours, nobody outside the platform team noticed, because the probe moved traffic to the healthy instance in seconds. That shift, from one routine failure equalling an outage to one routine failure equalling nothing, is the confidence signal this capability delivers, and it cost one extra small instance per service to reach.

Why this is a board-level reliability risk

Almost every avoidable outage of a load-balanced service comes back to the same two root causes: a backend pool with no redundancy, or a health check that did not reflect real application health. Both are cheap to fix and expensive to leave, which makes them exactly the kind of risk leadership should want closed before it is demonstrated by an incident.

The cost of getting this wrong is a customer-facing outage with a measurable revenue and reputation price, plus SLA credits if you promise uptime downstream. The cost of getting it right is a probe and, at most, one more instance per service. This is the rare class of reliability control where the downside of inaction is severe, the cost of action is small and bounded, and the trade should be quick to approve.

The leadership move on load-balancer resilience

The executive handle is not to approve each instance individually; it is to require that no production service runs on a single-instance pool and that every probe reflects real application health, by default and on the record.

1. Set a standard: no single-instance production pools

Ask for a date by which every production load balancer has at least two healthy instances spread across availability zones, and a standard that no new production load-balancing rule ships against a single-instance pool. A default enforced in the platform is the difference between hoping each team remembers redundancy and guaranteeing a single failure cannot take a service down.

2. Ask for proof the failover actually works

The proof is not a verbal 'it is redundant now.' It is an Advisor reliability view with no remaining single-instance pools on revenue services, plus a tested failover: reboot or drain one instance and confirm the service stays up. Microsoft's Load Balancer SLA only applies with at least two healthy instances, so 'two instances, tested' is the evidence to ask for.

3. Demand honest health checks, not just green ones

A probe that passes is not the same as a service that works. Require that each probe tests a real application health path, not a bare open port, so a green dashboard genuinely means a healthy service. The goal is not a load balancer that looks healthy; it is one that tells the truth when the application behind it is not.

Quick quiz

Question 1 of 5

A single VM rebooting for patches took a revenue service offline. As a leader, what end state should you be looking for as the defensible answer?

Keep learning

Go deeper on how Azure Load Balancer probes detect failure and how to design backend redundancy that actually survives one.

Two takeaways. Backend redundancy is the rare reliability control where the risk of inaction is a customer-facing outage and the cost of action is a free probe plus one small instance, so it is the trade to approve quickest. And the defensible end state is not 'the probes are green'; it is no production service on a single-instance pool, instances spread across availability zones, probes that test real application health, and a tested failover to prove one dead instance is a non-event.

Back to the library

Part of the learning path Build in Azure resilience

Configure load balancer health probes and backend redundancy

Health probes and backend redundancy: the basics

The probe that pointed at the wrong door

Finding single-instance pools and weak probes

How a probe decides an instance is healthydeep dive

What is the impact of a single-instance pool or a weak probe?

How do you make a load-balanced service survive a failure?

1. Inventory pools by instance count and probe quality

2. Add a second instance, spread across zones

3. Point the probe at real application health

4. Make redundancy the default in code

Quick quiz

Keep learning

Health probes and backend redundancy: the downtime view

The probe that pointed at the wrong door

How a finance partner frames the redundancy decision

Why a single-instance pool belongs on the risk register

What finance can do about the redundancy gap

1. Separate the free probe work from the per-instance cost

2. Order the work by downtime cost, not by Advisor count

3. Tie the second instance to the SLA you owe downstream

Quick quiz

Keep learning

Health probes and backend redundancy: the headline

The probe that pointed at the wrong door

What it looks like when one dead instance stops being an outage

Why this is a board-level reliability risk

The leadership move on load-balancer resilience

1. Set a standard: no single-instance production pools

2. Ask for proof the failover actually works

3. Demand honest health checks, not just green ones

Quick quiz

Keep learning

Related site reliability lessons