Fortinet black logo

OpenStack Administration Guide

Testing HA operation and failover

Copy Link
Copy Doc ID e842eb6e-694a-11ea-9384-00505692583a:209179
Download PDF

Testing HA operation and failover

This section describes how to verify that a FortiGate-VM HA cluster in an OpenStack environment is operating normally and will failover successfully.

On the cirros-l instance console (see the diagram in Deploying two FortiGate-VM instances in an HA configuration in an OpenStack environment), start a continuous ping to the IP address of cirros-r. On the cirros-r instance console, start a continuous ping to the IP address of cirros-l:

$ ping 172.32.0.11

PING 172.32.0.11 (172.32.0.11): 56 data bytes

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.402 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.433 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.502 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.408 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.362 ms

On both FortiGate-VMs, use the following diagnose command to sniff ICMP packets. You should only see packets going through the primary unit.

fgt-vm-1 # diagnose sniffer packet any 'icmp' 4

interfaces =[any]

filters= [icmp]

109.413710 port_ha in 169.251.0.1 - > 169.251.0.2: icmp: 169.251.0.1 udp port 53

unreachable

111.797651 port2 in 172.32.0.11 - > 172.33.0.12: icmp: echo request

111.797676 port3 out 172.33.0.1 - > 172.33.0.12: icmp: echo request

111.797932 port3 in 172.33.0.12 - > 172.33.0.1: icmp: echo reply

111.797910 port2 out 172.33.0.12 - > 172.32.0.11: icmp: echo reply

112.372066 port3 in 172.33.0.12 - > 172.32.0.11: icmp: echo request

112.372081 port2 out 172.32.0.9 - > 172.32.0.11: icmp: echo request

112.372225 port2 in 172.32.0.11 - > 172.32.0.9: icmp: echo reply

112.372232 port3 out 172.32.0.11 - > 172.33.0.12: icmp: echo reply

112.797831 port2 in 172.32.0.11 - > 172.33.0.12: icmp: echo request

112.797839 port3 out 172.33.0.1 - > 172.33.0.12: icmp: echo request

112.798019 port3 in 172.33.0.12 - > 172.33.0.1: icmp: echo reply

112.798021 port2 out 172.33.0.12 - > 172.32.0.11: icmp: echo reply

Shut down the primary unit. You can do this from the OpenStack Horizon Instances list.

After failover, enter the following diagnose command from the new primary unit to verify that the pings are now going through that unit:

fgt-vm-2 # diagnose sniffer packet any' icmp' 4

interfaces= [any]

filter s= [icmp]

0.360973 port3 in 172.33.0.12 - > 172.32.0.11: icmp: echo request

0.360983 port2 out 172.32.0.9 - > 172.32.0.11: icmp: echo request

0.361220 port2 in 172.32.0.11 - > 172.32.0.9: icmp: echo reply

0.361222 port3 out 172.32.0.11 - > 172.33.0.12: icmp: echo reply

0.785522 port2 in 172.32.0.11 - > 172.33.0.12: icmp: echo request

0.785527 port3 out 172.33.0.4 - > 172.33.0.12: icmp: echo request

0.785688 port3 in 172.33.0.12 - > 172.33.0.4: icmp: echo reply

0.785690 port2 out 172.33.0.12 - > 172.32.0.11: icmp: echo reply

1.360860 port3 in 172.33.0.12 - > 172.32.0.11: icmp: echo request

1.360864 port2 out 172.32.0.9 - > 172.32.0.11: icmp: echo request

1.361025 port2 in 172.32.0.11 - > 172.32.0.9: icmp: echo reply

1.361027 port3 out 172.32.0.11 - > 172.33.0.12: icmp: echo reply

Restart the FortiGate-VM instance that you shut down. After a short while it should re-join the cluster.

Testing HA operation and failover

This section describes how to verify that a FortiGate-VM HA cluster in an OpenStack environment is operating normally and will failover successfully.

On the cirros-l instance console (see the diagram in Deploying two FortiGate-VM instances in an HA configuration in an OpenStack environment), start a continuous ping to the IP address of cirros-r. On the cirros-r instance console, start a continuous ping to the IP address of cirros-l:

$ ping 172.32.0.11

PING 172.32.0.11 (172.32.0.11): 56 data bytes

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.402 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.433 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.502 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.408 ms

64 bytes from 172.32.0.11: seq=0 ttl=63 time=0.362 ms

On both FortiGate-VMs, use the following diagnose command to sniff ICMP packets. You should only see packets going through the primary unit.

fgt-vm-1 # diagnose sniffer packet any 'icmp' 4

interfaces =[any]

filters= [icmp]

109.413710 port_ha in 169.251.0.1 - > 169.251.0.2: icmp: 169.251.0.1 udp port 53

unreachable

111.797651 port2 in 172.32.0.11 - > 172.33.0.12: icmp: echo request

111.797676 port3 out 172.33.0.1 - > 172.33.0.12: icmp: echo request

111.797932 port3 in 172.33.0.12 - > 172.33.0.1: icmp: echo reply

111.797910 port2 out 172.33.0.12 - > 172.32.0.11: icmp: echo reply

112.372066 port3 in 172.33.0.12 - > 172.32.0.11: icmp: echo request

112.372081 port2 out 172.32.0.9 - > 172.32.0.11: icmp: echo request

112.372225 port2 in 172.32.0.11 - > 172.32.0.9: icmp: echo reply

112.372232 port3 out 172.32.0.11 - > 172.33.0.12: icmp: echo reply

112.797831 port2 in 172.32.0.11 - > 172.33.0.12: icmp: echo request

112.797839 port3 out 172.33.0.1 - > 172.33.0.12: icmp: echo request

112.798019 port3 in 172.33.0.12 - > 172.33.0.1: icmp: echo reply

112.798021 port2 out 172.33.0.12 - > 172.32.0.11: icmp: echo reply

Shut down the primary unit. You can do this from the OpenStack Horizon Instances list.

After failover, enter the following diagnose command from the new primary unit to verify that the pings are now going through that unit:

fgt-vm-2 # diagnose sniffer packet any' icmp' 4

interfaces= [any]

filter s= [icmp]

0.360973 port3 in 172.33.0.12 - > 172.32.0.11: icmp: echo request

0.360983 port2 out 172.32.0.9 - > 172.32.0.11: icmp: echo request

0.361220 port2 in 172.32.0.11 - > 172.32.0.9: icmp: echo reply

0.361222 port3 out 172.32.0.11 - > 172.33.0.12: icmp: echo reply

0.785522 port2 in 172.32.0.11 - > 172.33.0.12: icmp: echo request

0.785527 port3 out 172.33.0.4 - > 172.33.0.12: icmp: echo request

0.785688 port3 in 172.33.0.12 - > 172.33.0.4: icmp: echo reply

0.785690 port2 out 172.33.0.12 - > 172.32.0.11: icmp: echo reply

1.360860 port3 in 172.33.0.12 - > 172.32.0.11: icmp: echo request

1.360864 port2 out 172.32.0.9 - > 172.32.0.11: icmp: echo request

1.361025 port2 in 172.32.0.11 - > 172.32.0.9: icmp: echo reply

1.361027 port3 out 172.32.0.11 - > 172.33.0.12: icmp: echo reply

Restart the FortiGate-VM instance that you shut down. After a short while it should re-join the cluster.