Full mesh HA example

The following figure shows a full mesh HA configuration with a cluster of two FortiGates. This section describes the FortiGate configuration settings and network components required for a full mesh HA configuration. This section also contains example steps for setting up this full mesh HA configuration. The procedures in this section describe one of many possible sequences of steps for configuring full mesh HA. As you become more experienced with FortiOS, HA, and full mesh HA you may choose to use a different sequence of configuration steps.

Full Mesh HA configuration

For simplicity these procedures assume that you are starting with two new FortiGates set to the factory default configuration. However, starting from the default configuration is not a requirement for a successful HA deployment. FortiGate HA is flexible enough to support a successful configuration from many different starting points.

These procedures describe how to configure a cluster operating in NAT mode because NAT is the default FortiGate operating mode. However, the steps are the same if the cluster operates in transparent mode. You can either switch the cluster units to operate in transparent mode before beginning these procedures, or you can switch the cluster to operate in transparent mode after HA is configured and the cluster is connected and operating.

Full mesh HA configuration

The two FortiGates (FGT_ha_1 and FGT_ha_2) can be operating in NAT or transparent mode. Aside from the standard HA settings, the FortiGate configuration includes the following:

• The port5 and port6 interfaces configured as heartbeat interfaces. A full mesh HA configuration also includes redundant HA heartbeat interfaces.
• The port1 and port2 interfaces added to a redundant interface. Port1 is the active physical interface in this redundant interface. To make the port1 interface the active physical interface it should appear above the port2 interface in the redundant interface configuration.
• The port3 and port4 interfaces added to a redundant interface. Port3 is the active physical interface in this redundant interface. To make the port3 interface the active physical interface it should appear above the port4 interface in the redundant interface configuration.

Full mesh switch configuration

The following redundant switch configuration is required:

• Two redundant switches (Sw3 and Sw4) connected to the internal network. Establish an 802.1Q (Dot1Q) or interswitch-link (ISL) connection between them.
• Two redundant switches (Sw1 and Sw2) connected to the internet. Establish an 802.1Q (Dot1Q) or interswitch-link (ISL) connection between them.

Full mesh network connections

Make the following physical network connections for FGT_ha_1:

• Port1 to Sw1 (active)
• Port2 to Sw2 (inactive)
• Port3 to Sw3 (active)
• Port4 to Sw4 (inactive)

Make the following physical network connections for FGT_ha_2:

• Port1 to Sw2 (active)
• Port2 to Sw1 (inactive)
• Port3 to Sw4 (active)
• Port4 to Sw3 (inactive)

How packets travel from the internal network through the full mesh cluster and to the internet

If the cluster is operating in active-passive mode and FGT_ha_2 is the primary unit, all packets take the following path from the internal network to the internet:

1. From the internal network to Sw4. Sw4 is the active connection to FGT_ha_2; which is the primary unit. The primary unit receives all packets.
2. From Sw4 to the FGT_ha_2 port3 interface. Active connection between Sw4 and FGT_ha_2. Port3 is the active member of the redundant interface.
3. From FGT_ha_2 port3 to FGT_ha_2 port1. Active connection between FGT_ha_2 and Sw2. Port1 is the active member of the redundant interface.
4. From Sw2 to the external router and the internet.

Configuring full-mesh HA - GUI

Each cluster unit must have the same HA configuration.

To configure the FortiGates for HA operation

1. Register and apply licenses to the FortiGate.

   

2. On the System Information dashboard widget, beside Host Name select Change.

3. Enter a new Host Name for this FortiGate.

   New Name

   FGT_ha_1

4. Go toSystem > HA and change the following settings.

   Mode	Active-Active
   Group Name	Rexample1.com
   Password	RHA_pass_1
   Heartbeat Interface
   	Enable	Priority
   port5	Select	50
   port6	Select	50

5. Select OK.

   The FortiGate negotiates to establish an HA cluster. When you select OK you may temporarily lose connectivity with the FortiGate as the HA cluster negotiates and the FGCP changes the MAC address of the FortiGate interfaces. The MAC addresses of the FortiGate interfaces change to the following virtual MAC addresses:

   • port1 interface virtual MAC: 00-09-0f-09-00-00
   • port10 interface virtual MAC: 00-09-0f-09-00-01
   • port11 interface virtual MAC: 00-09-0f-09-00-02
   • port12 interface virtual MAC: 00-09-0f-09-00-03
   • port13 interface virtual MAC: 00-09-0f-09-00-04
   • port14 interface virtual MAC: 00-09-0f-09-00-05
   • port15 interface virtual MAC: 00-09-0f-09-00-06
   • port16 interface virtual MAC: 00-09-0f-09-00-07
   • port17 interface virtual MAC: 00-09-0f-09-00-08
   • port18 interface virtual MAC: 00-09-0f-09-00-09
   • port19 interface virtual MAC: 00-09-0f-09-00-0a
   • port2 interface virtual MAC: 00-09-0f-09-00-0b
   • port20 interface virtual MAC: 00-09-0f-09-00-0c
   • port3 interface virtual MAC: 00-09-0f-09-00-0d
   • port4 interface virtual MAC: 00-09-0f-09-00-0e
   • port5 interface virtual MAC: 00-09-0f-09-00-0f
   • port6 interface virtual MAC: 00-09-0f-09-00-10
   • port7 interface virtual MAC: 00-09-0f-09-00-11
   • port8 interface virtual MAC: 00-09-0f-09-00-12
   • port9 interface virtual MAC: 00-09-0f-09-00-13

   To be able to reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

   You can use the get hardware nic (or diagnose hardware deviceinfo nic) CLI command to view the virtual MAC address of any FortiGate interface. For example, use the following command to view the port1 interface virtual MAC address (Current_HWaddr) and the port1 permanent MAC address (Permanent_HWaddr):

   get hardware nic port1

   .

   .

   .

   MAC: 00:09:0f:09:00:00

   Permanent_HWaddr: 02:09:0f:78:18:c9

   .

   .

   .

6. Power off the first FortiGate.

7. Repeat these steps for the second FortiGate.

   Set the second FortiGate host name to:

   New Name

   FGT_ha_2

To connect the cluster to your network

1. Make the following physical network connections for FGT_ha_1:

   • Port1 to Sw1 (active)
   • Port2 to Sw2 (inactive)
   • Port3 to Sw3 (active)
   • Port4 to Sw4 (inactive)

2. Make the following physical network connections for FGT_ha_2:

   • Port1 to Sw2 (active)
   • Port2 to Sw1 (inactive)
   • Port3 to Sw4 (active)
   • Port4 to Sw3 (inactive)
3. Connect Sw3 and Sw4 to the internal network.
4. Connect Sw1 and Sw2 to the external router.
5. Enable 802.1Q (Dot1Q) or ISL communication between Sw1 and Sw2 and between Sw3 and Sw4.

6. Power on the cluster units.

   The units start and negotiate to choose the primary unit and the subordinate unit. This negotiation occurs with no user intervention.

   When negotiation is complete the cluster is ready to be configured for your network.

To view cluster status

Use the following steps to view the cluster dashboard and cluster members list to confirm that the cluster units are operating as a cluster.

1. View the system dashboard.

   The System Information dashboard widget shows the Cluster Name (Rexample1.com) and the host names and serial numbers of the Cluster Members. The Unit Operation widget shows multiple cluster units.

2. Go to System > HA to view the cluster members list.

   The list shows two cluster units, their host names, their roles in the cluster, and their priorities. You can use this list to confirm that the cluster is operating normally.

To troubleshoot the cluster configuration

If the cluster members list and the dashboard does not display information for both cluster units the FortiGates are not functioning as a cluster. See Full mesh HA example to troubleshoot the cluster.

To add basic configuration settings and the redundant interfaces

Use the following steps to add a few basic configuration settings.

1. Log into the cluster GUI.
2. Go to System > Administrators.
3. Edit admin and select Change Password.
4. Enter and confirm a new password.
5. Select OK.

6. Go to Network > Static Routes and temporarily delete the default route.

   You cannot add an interface to a redundant interface if any settings (such as the default route) are configured for it.

7. Go to Network > Interfaces and select Create New > Interface and configure the redundant interface to connect to the internet.

   Name	Port1_Port2
   Type	Redundant
   Physical Interface Members
   Selected Interfaces	port1, port2
   IP/Netmask	172.20.120.141/24

8. Select OK.

9. Select Create New and configure the redundant interface to connect to the internal network.

   Name	Port3_Port4
   Type	Redundant
   Physical Interface Members
   Selected Interfaces	port3, port4
   IP/Netmask	10.11.101.100/24
   Administrative Access	HTTPS, PING, SSH

10. Select OK.

    The virtual MAC addresses of the FortiGate interfaces change to the following. Notice that port1 and port2 both have the port1 virtual MAC address and port3 and port4 both have the port3 virtual MAC address:

    • port1 interface virtual MAC: 00-09-0f-09-00-00
    • port10 interface virtual MAC: 00-09-0f-09-00-01
    • port11 interface virtual MAC: 00-09-0f-09-00-02
    • port12 interface virtual MAC: 00-09-0f-09-00-03
    • port13 interface virtual MAC: 00-09-0f-09-00-04
    • port14 interface virtual MAC: 00-09-0f-09-00-05
    • port15 interface virtual MAC: 00-09-0f-09-00-06
    • port16 interface virtual MAC: 00-09-0f-09-00-07
    • port17 interface virtual MAC: 00-09-0f-09-00-08
    • port18 interface virtual MAC: 00-09-0f-09-00-09
    • port19 interface virtual MAC: 00-09-0f-09-00-0a
    • port2 interface virtual MAC: 00-09-0f-09-00-00 (same as port1)
    • port20 interface virtual MAC: 00-09-0f-09-00-0c
    • port3 interface virtual MAC: 00-09-0f-09-00-0d
    • port4 interface virtual MAC: 00-09-0f-09-00-0d (same as port3)
    • port5 interface virtual MAC: 00-09-0f-09-00-0f
    • port6 interface virtual MAC: 00-09-0f-09-00-10
    • port7 interface virtual MAC: 00-09-0f-09-00-11
    • port8 interface virtual MAC: 00-09-0f-09-00-12
    • port9 interface virtual MAC: 00-09-0f-09-00-13
11. Go to Router > Static > Static Routes.

12. Add the default route.

    Destination IP/Mask	0.0.0.0/0.0.0.0
    Gateway	172.20.120.2
    Device	Port1_Port2
    Distance	10

13. Select OK.

To configure HA port monitoring for the redundant interfaces

1. Go to System > HA.
2. In the cluster members list, edit the primary unit.
3. Enable interface monitoring the Port1_Port2 and the Port3_Port4 interfaces
4. Select OK.

Configuring full mesh HA - CLI

Each cluster must have the same HA configuration. Use the following procedure to configure the FortiGates for HA operation.

To configure the FortiGates for HA operation

1. Register and apply licenses to the FortiGate.

2. Enter a new Host Name for this FortiGate.

   config system global

   set hostname FGT_ha_1

   end

3. Configure HA settings.

   config system ha

   set mode a-a

   set group-name Rexample1.com

   set password RHA_pass_1

   set hbdev port5 50 port6 50

   end

   The FortiGate negotiates to establish an HA cluster. When you select OK you may temporarily lose connectivity with the FortiGate as the HA cluster negotiates and the FGCP changes the MAC address of the FortiGate interfaces. The MAC addresses of the FortiGate interfaces change to the following virtual MAC addresses:

   • port1 interface virtual MAC: 00-09-0f-09-00-00
   • port10 interface virtual MAC: 00-09-0f-09-00-01
   • port11 interface virtual MAC: 00-09-0f-09-00-02
   • port12 interface virtual MAC: 00-09-0f-09-00-03
   • port13 interface virtual MAC: 00-09-0f-09-00-04
   • port14 interface virtual MAC: 00-09-0f-09-00-05
   • port15 interface virtual MAC: 00-09-0f-09-00-06
   • port16 interface virtual MAC: 00-09-0f-09-00-07
   • port17 interface virtual MAC: 00-09-0f-09-00-08
   • port18 interface virtual MAC: 00-09-0f-09-00-09
   • port19 interface virtual MAC: 00-09-0f-09-00-0a
   • port2 interface virtual MAC: 00-09-0f-09-00-0b
   • port20 interface virtual MAC: 00-09-0f-09-00-0c
   • port3 interface virtual MAC: 00-09-0f-09-00-0d
   • port4 interface virtual MAC: 00-09-0f-09-00-0e
   • port5 interface virtual MAC: 00-09-0f-09-00-0f
   • port6 interface virtual MAC: 00-09-0f-09-00-10
   • port7 interface virtual MAC: 00-09-0f-09-00-11
   • port8 interface virtual MAC: 00-09-0f-09-00-12
   • port9 interface virtual MAC: 00-09-0f-09-00-13

   To be able to reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

   You can use the get hardware nic (or diagnose hardware deviceinfo nic) CLI command to view the virtual MAC address of any FortiGate interface. For example, use the following command to view the port1 interface virtual MAC address (Current_HWaddr) and the port1 permanent MAC address (Permanent_HWaddr):

   get hardware nic port1

   .

   .

   .

   MAC: 00:09:0f:09:00:00

   Permanent_HWaddr: 02:09:0f:78:18:c9

   .

   .

   .

4. Power off the first FortiGate.

5. Repeat these steps for the second FortiGate.

   Set the other FortiGate host name to:

   config system global

   set hostname FGT_ha_2

   end

To connect the cluster to your network

1. Make the following physical network connections for FGT_ha_1:

   • Port1 to Sw1 (active)
   • Port2 to Sw2 (inactive)
   • Port3 to Sw3 (active)
   • Port4 to Sw4 (inactive)

2. Make the following physical network connections for FGT_ha_2:

   • Port1 to Sw2 (active)
   • Port2 to Sw1 (inactive)
   • Port3 to Sw4 (active)
   • Port4 to Sw3 (inactive)
3. Connect Sw3 and Sw4 to the internal network.
4. Connect Sw1 and Sw2 to the external router.
5. Enable 802.1Q (Dot1Q) or ISL communication between Sw1 and Sw2 and between Sw3 and Sw4.

6. Power on the cluster units.

   The units start and negotiate to choose the primary unit and the subordinate unit. This negotiation occurs with no user intervention.

   When negotiation is complete the cluster is ready to be configured for your network.

To view cluster status

Use the following steps to view cluster status from the CLI.

1. Log into the CLI.

2. Enter get system status to verify the HA status of the cluster unit that you logged into.

   If the command output includes Current HA mode: a-a, master, the cluster units are operating as a cluster and you have connected to the primary unit.

   If the command output includes Current HA mode: a-a, backup, you have connected to a subordinate unit.

   If the command output includes Current HA mode: standalone the cluster unit is not operating in HA mode.

3. Enter the following command to confirm the HA configuration of the cluster:

   get system ha status
   HA Health Status: OK
   Model: FortiGate-XXXX
   Mode: HA A-P
   Group: 0
   Debug: 0
   Cluster Uptime: 7 days 00:30:26
   .
   .
   .
   

   You can use this command to confirm that the cluster is healthy and operating normally, some information about the cluster configuration, and information about how long the cluster has been operating. Information not shown in this example includes how the primary unit was selected, configuration synchronization status, usage stats for each cluster unit, heartbeat status, and the relative priorities of the cluster units.

4. Use the execute ha manage command to connect to the other cluster unit’s CLI and use these commands to verify cluster status.

To troubleshoot the cluster configuration

If the cluster members list and the dashboard does not display information for both cluster units the FortiGates are not functioning as a cluster. See Full mesh HA example to troubleshoot the cluster.

To add basic configuration settings and the redundant interfaces

Use the following steps to add a few basic configuration settings. Some steps use the CLI and some the GUI.

1. Log into the cluster CLI.

2. Add a password for the admin administrative account.

   config system admin

   edit admin

   set password <password_str>

   end

3. Temporarily delete the default route.

   You cannot add an interface to a redundant interface if any settings (such as the default route) are configured for it.

   config router static

   delete 1

   end

4. Go to System > Network > Interface and select Create New to add the redundant interface to connect to the internet.

5. Add the redundant interface to connect to the internet.

   config system interface

   edit Port1_Port2

   set type redundant

   set member port1 port2

   end

6. Add the redundant interface to connect to the internal network.

   config system interface

   edit Port3_Port4

   set type redundant

   set member port3 port4

   end

   The virtual MAC addresses of the FortiGate interfaces change to the following. Note that port1 and port2 both have the port1 virtual MAC address and port3 and port4 both have the port3 virtual MAC address:

   • port1 interface virtual MAC: 00-09-0f-09-00-00
   • port10 interface virtual MAC: 00-09-0f-09-00-01
   • port11 interface virtual MAC: 00-09-0f-09-00-02
   • port12 interface virtual MAC: 00-09-0f-09-00-03
   • port13 interface virtual MAC: 00-09-0f-09-00-04
   • port14 interface virtual MAC: 00-09-0f-09-00-05
   • port15 interface virtual MAC: 00-09-0f-09-00-06
   • port16 interface virtual MAC: 00-09-0f-09-00-07
   • port17 interface virtual MAC: 00-09-0f-09-00-08
   • port18 interface virtual MAC: 00-09-0f-09-00-09
   • port19 interface virtual MAC: 00-09-0f-09-00-0a
   • port2 interface virtual MAC: 00-09-0f-09-00-00 (same as port1)
   • port20 interface virtual MAC: 00-09-0f-09-00-0c
   • port3 interface virtual MAC: 00-09-0f-09-00-0d
   • port4 interface virtual MAC: 00-09-0f-09-00-0d (same as port3)
   • port5 interface virtual MAC: 00-09-0f-09-00-0f
   • port6 interface virtual MAC: 00-09-0f-09-00-10
   • port7 interface virtual MAC: 00-09-0f-09-00-11
   • port8 interface virtual MAC: 00-09-0f-09-00-12
   • port9 interface virtual MAC: 00-09-0f-09-00-13
7. Go to Router > Static > Static Routes.

8. Add the default route.

   config router static

   edit 1

   set dst 0.0.0.0 0.0.0.0

   set gateway 172.20.120.2

   set device Port1_Port2

   end

To configure HA port monitoring for the redundant interfaces

1. Enter the following command to configure port monitoring for the redundant interfaces:

   config system ha

   set monitor Port1_Port2 Port3_Port4

   end