HA with redundant interfaces

On FortiGate models that support it you can combine two or more interfaces into a single redundant interface. A redundant interface consists of two or more physical interfaces. Traffic is processed by the first physical interface in the redundant interface. If that physical interface fails, traffic fails over to the next physical interface. Redundant interfaces don’t have the benefit of improved performance that aggregate interfaces can have, but they do provide failover if a physical interface fails or is disconnected.

Example cluster with a redundant interfaces

This example describes how to configure an HA cluster consisting of two FortiGates with a redundant interface connection to the internet and to an internal network. The connection to the internet uses port1 and port2. The connection to the internal network uses port3 and port4. The HA heartbeat uses port5 and port6.

The redundant interfaces are also configured as HA monitored interfaces.

HA interface monitoring, link failover, and redundant interfaces

HA interface monitoring monitors the redundant interface as a single interface and does not monitor the individual physical interfaces in the redundant interface. HA interface monitoring registers the redundant interface to have failed only if all the physical interfaces in the redundant interface have failed. If only some of the physical interfaces in the redundant interface fail or become disconnected, HA considers the redundant interface to be operating normally.

HA MAC addresses and redundant interfaces

For a standalone FortiGate a redundant interface has the MAC address of the first physical interface added to the redundant interface configuration. A redundant interface consisting of port1 and port2 would have the MAC address of port1.

In an HA cluster, HA changes the MAC addresses of the cluster interfaces to virtual MAC addresses. A redundant interface in a cluster acquires the virtual MAC address that would have been acquired by the first physical interface added to the redundant interface configuration.

Connecting multiple redundant interfaces to one switch while operating in active-passive HA mode

HA assigns the same virtual MAC addresses to the subordinate unit interfaces as are assigned to the corresponding primary unit interfaces. Consider a cluster of two FortiGates operating in active-passive mode with a redundant interface consisting of port1 and port2. You can connect multiple redundant interfaces to the same switch if you configure the switch so that it defines multiple separate redundant interfaces and puts the redundant interfaces of each cluster unit into separate redundant interfaces. In this configuration, each cluster unit forms a separate redundant interface with the switch.

However, if the switch is configured with a single four-port redundant interface configuration, because the same MAC addresses are being used by both cluster units, the switch adds all four interfaces (port1 and port2 from the primary unit and port1 and port2 from the subordinate unit) to the same redundant interface.

To avoid unpredictable results, when you connect a switch to multiple redundant interfaces in an active-passive cluster you should configure separate redundant interfaces on the switch; one for each cluster unit.

Connecting multiple redundant interfaces to one switch while operating in active-active HA mode

In an active-active cluster, all cluster units send and receive packets. To operate a cluster with redundant interfaces in active-active mode, with multiple redundant interfaces connected to the same switch, you must separate the redundant interfaces of each cluster unit into different redundant interfaces on the connecting switch.

General configuration steps

The section includes GUI and CLI procedures. These procedures assume that the FortiGates are running the same FortiOS firmware build and are set to the factory default configuration.

General configuration steps

1. Apply licenses to the FortiGates to become the cluster.

2. Configure the FortiGates for HA operation.

   • Change each unit’s host name.
   • Configure HA.
3. Connect the cluster to the network.
4. View cluster status.

5. Add basic configuration settings and configure the redundant interfaces.

   • Add a password for the admin administrative account.
   • Add the redundant interfaces.
   • Add a default route.

   You could also configure redundant interfaces in each FortiGate before they form a cluster.

6. Configure HA port monitoring for the redundant interfaces.

Configuring active-passive HA cluster that includes redundant interfaces - GUI

These procedures assume you are starting with two FortiGates with factory default settings.

To configure the FortiGates for HA operation

1. Register and apply licenses to the FortiGate.

   

2. On the System Information dashboard widget, beside Host Name select Change.

3. Enter a new Host Name for this FortiGate.

   New Name

   FGT_ha_1

4. Select OK.

5. Go to System > HA and change the following settings.

   Mode	Active-Passive
   Group Name	example6.com
   Password	HA_pass_6
   Heartbeat Interface
   	Enable	Priority
   port5	Select	50
   port6	Select	50

   Since port3 and port4 will be used for a redundant interface, you must change the HA heartbeat configuration.

6. Select OK.

   The FortiGate negotiates to establish an HA cluster. When you select OK you may temporarily lose connectivity with the FortiGate as the HA cluster negotiates and the FGCP changes the MAC address of the FortiGate interfaces. The MAC addresses of the FortiGate interfaces change to the following virtual MAC addresses:

   • port1 interface virtual MAC: 00-09-0f-09-00-00
   • port10 interface virtual MAC: 00-09-0f-09-00-01
   • port11 interface virtual MAC: 00-09-0f-09-00-02
   • port12 interface virtual MAC: 00-09-0f-09-00-03
   • port13 interface virtual MAC: 00-09-0f-09-00-04
   • port14 interface virtual MAC: 00-09-0f-09-00-05
   • port15 interface virtual MAC: 00-09-0f-09-00-06
   • port16 interface virtual MAC: 00-09-0f-09-00-07
   • port17 interface virtual MAC: 00-09-0f-09-00-08
   • port18 interface virtual MAC: 00-09-0f-09-00-09
   • port19 interface virtual MAC: 00-09-0f-09-00-0a
   • port2 interface virtual MAC: 00-09-0f-09-00-0b
   • port20 interface virtual MAC: 00-09-0f-09-00-0c
   • port3 interface virtual MAC: 00-09-0f-09-00-0d
   • port4 interface virtual MAC: 00-09-0f-09-00-0e
   • port5 interface virtual MAC: 00-09-0f-09-00-0f
   • port6 interface virtual MAC: 00-09-0f-09-00-10
   • port7 interface virtual MAC: 00-09-0f-09-00-11
   • port8 interface virtual MAC: 00-09-0f-09-00-12
   • port9 interface virtual MAC: 00-09-0f-09-00-13

   To reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

   You can use the get hardware nic (or diagnose hardware deviceinfo nic) CLI command to view the virtual MAC address of any FortiGate interface. For example, use the following command to view the port1 interface virtual MAC address (Current_HWaddr) and the port1 permanent MAC address (Permanent_HWaddr):

   get hardware nic port1

   .

   .

   .

   MAC: 00:09:0f:09:00:00

   Permanent_HWaddr: 02:09:0f:78:18:c9

   .

   .

   .

7. Power off the first FortiGate.

8. Repeat these steps for the second FortiGate.

   Set the second FortiGate host name to:

   New Name

   FGT_ha_2

To connect the cluster to the network

1. Connect the port1 and port2 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the internet.

   Configure the switch so that the port1 and port2 of FGT_ha_1 make up a redundant interface and port1 and port2 of FGT_ha_2 make up another redundant interface.

2. Connect the port3 and port4 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the internal network.

   Configure the switch so that the port3 and port4 of FGT_ha_1 make up a redundant interface and port3 and port4 of FGT_ha_2 make up another redundant interface.

3. Connect the port5 interfaces of FGT_ha_1 and FGT_ha_2 together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.
4. Connect the port5 interfaces of the cluster units together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.

5. Power on the cluster units.

   The units start and negotiate to choose the primary unit and the subordinate unit. This negotiation occurs with no user intervention and normally takes less than a minute.

   When negotiation is complete the cluster is ready to be configured for your network.

To view cluster status

Use the following steps to view the cluster dashboard and cluster members list to confirm that the cluster units are operating as a cluster.

1. View the system dashboard.

   The HA Status dashboard widget displays how long the cluster has been operating (Uptime) and the time since the last failover occurred (State Changed). You can hover over the State Changed time to see the event that caused the state change. You can also click on the HA Status dashboard widget to configure HA settings or to get a listing of the most recent HA events recorded by the cluster.

2. Go to System > HA to view the cluster members list.

   The list shows both cluster units, their host names, their roles in the cluster, and their device priorities. You can use this list to confirm that the cluster is operating normally. For example, if the list shows only one cluster unit then the other unit has left the cluster for some reason.

To troubleshoot the cluster configuration

See Troubleshooting.

To add basic configuration settings and the redundant interfaces

Use the following steps to add a few basic configuration settings.

1. Log into the cluster GUI.
2. Go to System > Administrators.
3. Edit admin and select Change Password.
4. Enter and confirm a new password.
5. Select OK.

6. Go to Network > Static Routes and temporarily delete the default route.

   You cannot add an interface to a redundant interface if any settings (such as the default route) are configured for it.

7. Go to Network > Interfaces and select Create New > Interface to add the redundant interface to connect to the internet.

8. Set Type to Redundant Interface and configure the redundant interface to be connected to the internet:

   Name	Port1_Port2
   Physical Interface Members	port1, port2
   IP/Netmask	172.20.120.141/24

9. Select OK.
10. Select Create New to add the redundant interface to connect to the internal network.

11. Set Type to Redundant Interface and configure the redundant interface to be connected to the internet:

    Name	Port3_Port4
    Physical Interface Members	port3, port4
    IP/Netmask	10.11.101.100/24
    Administrative Access	HTTPS, PING, SSH

12. Select OK.

    The virtual MAC addresses of the FortiGate interfaces change to the following. Note that port1 and port2 both have the port1 virtual MAC address and port3 and port4 both have the port3 virtual MAC address:

    • port1 interface virtual MAC: 00-09-0f-09-00-00
    • port10 interface virtual MAC: 00-09-0f-09-00-01
    • port11 interface virtual MAC: 00-09-0f-09-00-02
    • port12 interface virtual MAC: 00-09-0f-09-00-03
    • port13 interface virtual MAC: 00-09-0f-09-00-04
    • port14 interface virtual MAC: 00-09-0f-09-00-05
    • port15 interface virtual MAC: 00-09-0f-09-00-06
    • port16 interface virtual MAC: 00-09-0f-09-00-07
    • port17 interface virtual MAC: 00-09-0f-09-00-08
    • port18 interface virtual MAC: 00-09-0f-09-00-09
    • port19 interface virtual MAC: 00-09-0f-09-00-0a
    • port2 interface virtual MAC: 00-09-0f-09-00-00 (same as port1)
    • port20 interface virtual MAC: 00-09-0f-09-00-0c
    • port3 interface virtual MAC: 00-09-0f-09-00-0d
    • port4 interface virtual MAC: 00-09-0f-09-00-0d (same as port3)
    • port5 interface virtual MAC: 00-09-0f-09-00-0f
    • port6 interface virtual MAC: 00-09-0f-09-00-10
    • port7 interface virtual MAC: 00-09-0f-09-00-11
    • port8 interface virtual MAC: 00-09-0f-09-00-12
    • port9 interface virtual MAC: 00-09-0f-09-00-13
13. Go to Router > Static > Static Routes.

14. Add the default route.

    Destination IP/Mask	0.0.0.0/0.0.0.0
    Gateway	172.20.120.2
    Device	Port1_Port2
    Distance	10

15. Select OK.

To configure HA port monitoring for the redundant interfaces

1. Go to System > HA.
2. In the cluster members list, edit the primary unit.

3. Configure the following port monitoring for the redundant interfaces:

   	Port Monitor
   Port1_Port2	Select
   Port3_Port4	Select

4. Select OK.

Configuring active-passive HA cluster that includes redundant interfaces - CLI

These procedures assume you are starting with two FortiGates with factory default settings.

To configure the FortiGates for HA operation

1. Register and apply licenses to the FortiGate. This includes FortiCloud activation and FortiClient licensing, and entering a license key if you purchased more than 10 Virtual Domains (VDOMS). All of the FortiGates in a cluster must have the same level of licensing.
2. You can also install any third-party certificates on the primary FortiGate before forming the cluster. Once the cluster is formed third-party certificates are synchronized to the backup FortiGate.
   We recommend that you add FortiToken licenses and FortiTokens to the primary unit after the cluster has formed.

3. Change the host name for this FortiGate:

   config system global

   set hostname FGT_ha_1

   end

4. Configure HA settings.

   config system ha

   set mode a-p

   set group-name example6.com

   set password HA_pass_6

   set hbdev port5 50 port6 50

   end

   Since port3 and port4 will be used for a redundant interface, you must change the HA heartbeat configuration.

   The FortiGate negotiates to establish an HA cluster. You may temporarily lose connectivity with the FortiGate as the HA cluster negotiates and the FGCP changes the MAC address of the FortiGate interfaces. The MAC addresses of the FortiGate interfaces change to the following virtual MAC addresses:

   • port1 interface virtual MAC: 00-09-0f-09-00-00
   • port10 interface virtual MAC: 00-09-0f-09-00-01
   • port11 interface virtual MAC: 00-09-0f-09-00-02
   • port12 interface virtual MAC: 00-09-0f-09-00-03
   • port13 interface virtual MAC: 00-09-0f-09-00-04
   • port14 interface virtual MAC: 00-09-0f-09-00-05
   • port15 interface virtual MAC: 00-09-0f-09-00-06
   • port16 interface virtual MAC: 00-09-0f-09-00-07
   • port17 interface virtual MAC: 00-09-0f-09-00-08
   • port18 interface virtual MAC: 00-09-0f-09-00-09
   • port19 interface virtual MAC: 00-09-0f-09-00-0a
   • port2 interface virtual MAC: 00-09-0f-09-00-0b
   • port20 interface virtual MAC: 00-09-0f-09-00-0c
   • port3 interface virtual MAC: 00-09-0f-09-00-0d
   • port4 interface virtual MAC: 00-09-0f-09-00-0e
   • port5 interface virtual MAC: 00-09-0f-09-00-0f
   • port6 interface virtual MAC: 00-09-0f-09-00-10
   • port7 interface virtual MAC: 00-09-0f-09-00-11
   • port8 interface virtual MAC: 00-09-0f-09-00-12
   • port9 interface virtual MAC: 00-09-0f-09-00-13

   To reconnect sooner, you can update the ARP table of your management PC by deleting the ARP table entry for the FortiGate (or just deleting all arp table entries). You may be able to delete the arp table of your management PC from a command prompt using a command similar to arp -d.

   You can use the get hardware nic (or diagnose hardware deviceinfo nic) CLI command to view the virtual MAC address of any FortiGate interface. For example, use the following command to view the port1 interface virtual MAC address (Current_HWaddr) and the port1 permanent MAC address (Permanent_HWaddr):

   get hardware nic port1

   .

   .

   .

   MAC: 00:09:0f:09:00:00

   Permanent_HWaddr: 02:09:0f:78:18:c9

   .

   .

   .

5. Repeat these steps for the other FortiGate.

   Set the other FortiGate host name to:

   config system global

   set hostname FGT_ha_2

   end

To connect the cluster to the network

1. Connect the port1 and port2 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the internet.

   Configure the switch so that the port1 and port2 of FGT_ha_1 make up a redundant interface and port1 and port2 of FGT_ha_2 make up another redundant interface.

2. Connect the port3 and port4 interfaces of FGT_ha_1 and FGT_ha_2 to a switch connected to the internal network.

   Configure the switch so that the port3 and port4 of FGT_ha_1 make up a redundant interface and port3 and port4 of FGT_ha_2 make up another redundant interface.

3. Connect the port5 interfaces of FGT_ha_1 and FGT_ha_2 together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.
4. Connect the port5 interfaces of the cluster units together. You can use a crossover Ethernet cable or regular Ethernet cables and a switch.

5. Power on the cluster units.

   The units start and negotiate to choose the primary unit and the subordinate unit. This negotiation occurs with no user intervention and normally takes less than a minute.

   When negotiation is complete the cluster is ready to be configured for your network.

To view cluster status

Use the following steps to view cluster status from the CLI.

1. Log into the CLI.

2. Enter get system status to verify the HA status of the cluster unit that you logged into.Look for the following information in the command output.

   Current HA mode: a-a, master	The cluster units are operating as a cluster and you have connected to the primary unit.
   Current HA mode: a-a, backup	The cluster units are operating as a cluster and you have connected to a subordinate unit.
   Current HA mode: standalone	The cluster unit is not operating in HA mode

3. Enter the following command to confirm the HA configuration of the cluster:

   get system ha status
   HA Health Status: OK
   Model: FortiGate-XXXX
   Mode: HA A-P
   Group: 0
   Debug: 0
   Cluster Uptime: 7 days 00:30:26
   .
   .
   .

   You can use this command to confirm that the cluster is healthy and operating normally, some information about the cluster configuration, and information about how long the cluster has been operating. Information not shown in this example includes how the primary unit was selected, configuration synchronization status, usage stats for each cluster unit, heartbeat status, and the relative priorities of the cluster units.

To troubleshoot the cluster configuration

See Troubleshooting.

To add basic configuration settings and the redundant interfaces

Use the following steps to add a few basic configuration settings and the redundant interfaces.

1. Add a password for the admin administrative account.

   config system admin

   edit admin

   set password <psswrd>

   end

2. Temporarily delete the default route.

   You cannot add an interface to a redundant interface if any settings (such as the default route) are configured for it. In this example the index of the default route is 1.

   config router static

   delete 1

   end

3. Add the redundant interfaces:

   config system interface

   edit Port1_Port2

   set type redundant

   set member port1 port2

   set ip 172.20.120.141/24

   set vdom root

   next

   edit Port3_Port4

   set type redundant

   set member port3 port4

   set ip 10.11.101.100/24

   set vdom root

   end

   The virtual MAC addresses of the FortiGate interfaces change to the following. Note that port1 and port2 both have the port1 virtual MAC address and port3 and port4 both have the port3 virtual MAC address:

   • port1 interface virtual MAC: 00-09-0f-09-00-00
   • port10 interface virtual MAC: 00-09-0f-09-00-01
   • port11 interface virtual MAC: 00-09-0f-09-00-02
   • port12 interface virtual MAC: 00-09-0f-09-00-03
   • port13 interface virtual MAC: 00-09-0f-09-00-04
   • port14 interface virtual MAC: 00-09-0f-09-00-05
   • port15 interface virtual MAC: 00-09-0f-09-00-06
   • port16 interface virtual MAC: 00-09-0f-09-00-07
   • port17 interface virtual MAC: 00-09-0f-09-00-08
   • port18 interface virtual MAC: 00-09-0f-09-00-09
   • port19 interface virtual MAC: 00-09-0f-09-00-0a
   • port2 interface virtual MAC: 00-09-0f-09-00-00 (same as port1)
   • port20 interface virtual MAC: 00-09-0f-09-00-0c
   • port3 interface virtual MAC: 00-09-0f-09-00-0d
   • port4 interface virtual MAC: 00-09-0f-09-00-0d (same as port3)
   • port5 interface virtual MAC: 00-09-0f-09-00-0f
   • port6 interface virtual MAC: 00-09-0f-09-00-10
   • port7 interface virtual MAC: 00-09-0f-09-00-11
   • port8 interface virtual MAC: 00-09-0f-09-00-12
   • port9 interface virtual MAC: 00-09-0f-09-00-13

4. Add the default route.

   config router static

   edit 1

   set dst 0.0.0.0 0.0.0.0

   set gateway 172.20.120.2

   set device Port1_Port2

   end

To configure HA port monitoring for the redundant interfaces

1. Configure HA port monitoring for the redundant interfaces.

   config system ha

   set monitor Port1_Port2 Port3_Port4

   end