Configuring SD-WAN in an HA cluster using internal hardware switches

In this SD-WAN configuration, two FortiGates in an active-passive (A-P) HA pair are used to provide hardware redundancy. Instead of using external switches to provide a mesh network connection to the ISP routers, the FortiGates use their built-in hardware switches to connect to the ISP routers.

Only FortiGate models that have hardware switches can be used for this solution. Ports in a software switch are not in a forwarding state when a FortiGate is acting as a secondary device in a A-P cluster.

In this topology:

• Two hardware switches are created, HD_SW1 and HD_SW2.

• HD_SW1 is used to connect to ISP 1 Router and includes the internal1 and internal2 ports.

• HD_SW2 is used to connect to ISP 2 Router and includes the internal3 and internal4 ports.

• Another interface on each device is used as the HA heartbeat interface, connecting the two FortiGates in HA.

The FortiGates create two hardware switches to connect to ISP 1 and ISP2. When FGT_A is the primary device, it reaches ISP 1 on internal1 in HD_SW1 and ISP 2 on internal4 in HD_SW2. When FGT_B is the primary device, it reaches ISP 1 on internal2 in HD_SW1 and ISP 2 on internal3 on HD_SW2.

HA failover

This is not a standard HA configuration with external switches. In the case of a device failure, one of the ISPs will no longer be available because the switch that is connected to it will be down.

For example, If FGT_A loses power, HA failover will occur and FGT_B will become the primary unit. Its connection to internal2 on HD_SW1 will also be down, so it will be unable to connect to ISP 1. Its SD-WAN SLAs will be broken, and traffic will only be routed through ISP 2.

A link on a hardware switch cannot be monitored in HA monitor, so it is impossible to perform link failure when a port in either of the hardware switches fails. Performing a link failure is unnecessary in this configuration though, because any link failure on the hardware switch will be experienced by both cluster members. SD-WAN SLA health checks should be used to monitor the health of each ISP.

Failure on a hardware switch or ISP router

If a hardware switch or switch interface is down, or the ISP router is down, the SD-WAN can detect the broken SLA and continue routing to the other ISP.

For example, if FGT_A is the primary unit, and ISP 2 Router becomes unreachable, the SLA health checks on SD-WAN will detect the broken SLA and cause traffic to stop routing to ISP 2.

Configuration

To configure the HA A-P cluster with internal hardware switches:

1. Configure two FortiGates with internal switches in an A-P HA cluster (follow the steps in HA active-passive cluster setup), starting by connecting the heartbeat interface.

2. When the HA cluster is up, connect to the primary FortiGate's GUI.

3. Remove the existing interface members from the default hardware switch:

   1. Go to Network > Interfaces.

   2. In the LAN section, double-click the internal interface to edit it.

   3. In Interface Members, remove all of the interfaces

      

   4. Click OK.

4. Configure the hardware switch interfaces for the two ISPs:

   1. Go to Network > Interfaces and click Create New > Interface.

   2. Enter a name (HD_SW1).

   3. Set Type to Hardware Switch.

   4. In Interface Members, add two interfaces (internal1 and internal2).

   5. Set IP/Netmask to 192.168.1.2/24.

   6. Configure the remaining settings as needed.

      

   7. Click OK.

   8. Repeat these steps to create a second hardware switch interface (HD_SW2) with two interface members (internal3 and internal4) and IP/Netmask set to 192.168.3.2/24.

      

To connect the devices as shown in the topology:

1. Connect the incoming interface to the internal switch on both FortiGates.

2. On FGT_A, connect internal1 of HD_SW1 to ISP 1 Router.

3. On FGT_B, connect internal3 of HD_SW2 to ISP 2 Router.

4. For HD_SW1, connect FGT_A internal2 directly to FGT_B internal2.

5. For HD_SW2, connect FGT_A internal4 directly to FGT_B internal4.

To configure SD-WAN:

The primary FortiGate makes all the SD-WAN decisions.

1. On the primary FortiGate, go to Network > SD-WAN, select the SD-WAN Zones tab, and click Create New > SD-WAN Member.

2. In the Interface dropdown, select HD_SW1.

3. Leave SD-WAN Zone set to virtual-wan-link.

4. Enter the Gateway address 192.168.1.1.

5. Click OK.

6. Repeat these steps to add the second interface (HD_SW2) with the gateway 192.168.3.1.

7. Click Apply.

   

8. Create a health check:

   1. Go to Network > SD-WAN, select the Performance SLA tab, and click Create New.

   2. Set Name to GW_HC.

   3. Set Protocol to Ping and Servers to 8.8.8.8.

   4. Set Participants to All SD-WAN Members.

   5. Enable SLA Target and leave the default values.

   6. Click OK.

9. Create SD-WAN rules as needed. The SLA health check can be used to determine when the ISP connections are in or out of SLA, and to failover accordingly.