CloudLabBox
Try the free demo

Palo Alto NGFW lab 28 of 30

Troubleshooting high availability: split brain, HA2 and a flapping firewall

A contractor left the Leeds pair broken. Both firewalls claim to be active, the session link is down, and once those are fixed the preferred firewall flaps and falls over. Read the HA state and system log, fix each layer in turn, synchronise the configuration, then prove the repair by failing over with a download running.

Run this lab

It needs Standard: its 4 nodes are set to 6 vCPU and 12,800 MB in total.

In your workspace: Shared, then Palo Alto NGFW PAN-OS 11 guided labs, then 28 - Troubleshooting high availability - split brain HA2 and a flapping firewall

Run it on Standard from £5 a day

The free demo can’t run it: it has 6 vCPU and 8 GB, and this lab’s nodes need 6 vCPU and 12,800 MB together. Every lab plan includes every guided lab.

Level
Advanced
Time
About 75 minutes
Nodes need
6 vCPU, 12,800 MB
Smallest plan
Standard

The scenario

A contractor installed the second firewall at the Leeds depot and left before testing it. The design on the wall is simple:

  • FW-A is preferred and normally active. FW-B is passive.
  • HA1 runs over the management ports. HA2 runs on the cable between the two ethernet1/3 ports.
  • Link monitoring watches ethernet1/1 and ethernet1/2. Path monitoring pings the ISP gateway, 203.0.113.1.
  • FW-A holds the current rulebase. FW-B should get it through configuration sync.

Users now drop off at random, and monitoring reports two active firewalls. There isn't one fault here. There are several, stacked up, and fixing one uncovers the next. Work up the layers, and let the firewalls tell you what's wrong before you change anything.

Before you start. Start all the nodes. Both firewalls take 6-9 minutes to boot and come up with this lab's starting configuration already loaded. Wait until each console shows its own name (FW-A login:, FW-B login:), then log in as admin / Lab-Access-1. Until the management plane is ready it answers Login incorrect, even with the right password. The Linux hosts are ready within seconds.

Commits on a pair that isn't in sync ask a question. Expect Would you like to proceed with commit? (y or n). Answer y. The warning is true: the commit applies to that firewall only.

What you’ll practise

  • Read the HA state, link, path and synchronisation outputs and tell which layer has failed
  • Recognise split brain and fix an HA1 peer that can't be reached
  • Fix an HA2 fault and explain why the peer waits in the initial state
  • Diagnose path-monitoring flaps from the system log, and recover a non-functional firewall
  • Synchronise the running configuration, and prove session synchronisation with a live failover

The tasks

  1. Learn to read the pair
  2. Two active firewalls
  3. The peer that stays in initial
  4. The preferred firewall falls over
  5. Synchronise the configuration
  6. Prove it with a live download

The Lab Tasks pane has the scenario, these tasks, the checks to run, hints and the full solution.

Topology and devices

The network as it opens in EVE-NG, drawn from the lab’s own canvas, and the image each node runs.

Topology of Troubleshooting high availability - split brain HA2 and a flapping firewall4 nodes and 2 networks, 9 links. Every link is listed as text below the diagram.FW-AFW-BClient-PCInternetInsideOutsideeth0eth0mgmtethernet1/2mgmtethernet1/2ethernet1/1ethernet1/1ethernet1/3ethernet1/3
4 nodes and 2 networks, with the interface names each link uses. On a narrow screen, scroll the diagram sideways.
Every link, as text
  • FW-A mgmt to Inside (network)
  • FW-B mgmt to Inside (network)
  • FW-A ethernet1/2 to Inside (network)
  • FW-B ethernet1/2 to Inside (network)
  • FW-A ethernet1/1 to Outside (network)
  • FW-B ethernet1/1 to Outside (network)
  • FW-A ethernet1/3 to FW-B ethernet1/3
  • Client-PC eth0 to Inside (network)
  • Internet eth0 to Outside (network)
Nodes in Troubleshooting high availability: split brain, HA2 and a flapping firewall, with the image each runs and the vCPU and RAM it’s set to
NodeDeviceImage in Add NodeSet to
FW-APalo Alto Networks VM-Series (PAN-OS)paloalto-11.2.52 vCPU, 5,632 MB
FW-BPalo Alto Networks VM-Series (PAN-OS)paloalto-11.2.52 vCPU, 5,632 MB
Client-PCEVE-NG GUI server (Docker)eve-gui-server:latest1 vCPU, 1,024 MB
InternetEVE-NG GUI server (Docker)eve-gui-server:latest1 vCPU, 512 MB

How to open this lab

  1. Sign in to your workspace at lab.cloudlabbox.com.

  2. In File Manager, open Shared, then the folder Palo Alto NGFW PAN-OS 11 guided labs.

  3. Open 28 - Troubleshooting high availability - split brain HA2 and a flapping firewall and start its nodes, then open Lab Tasks for the scenario and the steps.

The lab is read-only. You can start the nodes and configure the devices, but changes to the topology can’t be saved. To change it, rebuild it in your own folder.

More Palo Alto NGFW labs

All Palo Alto NGFW labs

Related labs in other courses

CloudLabBox is independent of Palo Alto Networks and Cisco. The lab follows exam topics and isn’t vendor training material; vendor names identify the exams and the images in the workspace.