Network topologies

The FortiGate unit requires only one active FortiLink to manage all of the subtending FortiSwitch units (called stacking).

You can configure the FortiLink as a physical interface or as a logical interface (associated with one or more physical interfaces). Depending on the network topology, you can also configure a standby FortiLink.

NOTE: For any of the topologies:

• All of the managed FortiSwitch units will function as one Layer-2 stack where the FortiGate unit manages each FortiSwitch separately.
• The active FortiLink carries data as well as management traffic.

Supported topologies

Fortinet recommends the following topologies for managed FortiSwitch units:

• Single FortiGate managing a single FortiSwitch unit
• Single FortiGate unit managing a stack of several FortiSwitch units
• HA-mode FortiGate units managing a single FortiSwitch unit
• HA-mode FortiGate units managing a stack of several FortiSwitch units
• HA-mode FortiGate units managing a FortiSwitch two-tier topology
• Single FortiGate unit managing multiple FortiSwitch units (using a hardware or software switch interface)
• HA-mode one-tier MCLAG
• Dual-homed servers connected to a pair of FortiSwitch units using an MCLAG
• Standalone FortiGate unit with dual-homed FortiSwitch access
• HA-mode FortiGate units with dual-homed FortiSwitch access
• Multi-tiered MCLAG with HA-mode FortiGate units
• Three-tier FortiLink MCLAG configuration
• HA-mode FortiGate units using hardware-switch interfaces and STP
• HA-mode FortiGate units in remote sites
• FortiLink with an HA cluster of four FortiGate units
• FortiLink over a point-to-point layer-2 network

Single FortiGate managing a single FortiSwitch unit

On the FortiGate unit, the FortiLink interface is configured as a physical or aggregate interface. The 802.3ad aggregate interface type provides a logical grouping of one or more physical interfaces.

NOTE:

• For the aggregate interface, you must disable the split interface on the FortiGate unit.
• When you are using the aggregate interface on the FortiGate unit for the FortiLink interface, the lacp-mode of the FortiLink aggregate interface must be set to static. Unless MCLAG is enabled and you are using 6.2.0 or later, see Transitioning from a FortiLink split interface to a FortiLink MCLAG for details.
• Do not create loops or rings with the FortiSwitch units because the FortiGate unit does not use the STP.

Single FortiGate unit managing a stack of several FortiSwitch units

The FortiGate unit connects directly to one FortiSwitch unit using a physical or aggregate interface. The remaining FortiSwitch units connect in a ring using inter-switch links (that is, ISL).

Optionally, you can connect a standby FortiLink connection to the last FortiSwitch unit. For this configuration, you create a FortiLink Split-Interface (an aggregate interface that contains one active link and one standby link).

NOTE:

• When you are using the aggregate interface on the FortiGate unit for the FortiLink interface, the lacp-mode of the FortiLink aggregate interface must be set to static. Unless MCLAG is enabled and you are using 6.2.0 or later, see Transitioning from a FortiLink split interface to a FortiLink MCLAG for details.
• External devices shown in the following topology must be compliant endpoints, such as computers. They cannot be third-party switches or appliances.
• Do not create loops or rings with the FortiGate unit in the path.

HA-mode FortiGate units managing a single FortiSwitch unit

The master and slave FortiGate units both connect a FortiLink to the FortiSwitch unit. The FortiLink port(s) and interface type must match on the two FortiGate units.

NOTE: Before FortiOS 6.2.0, when using HA-mode FortiGate units to manage FortiSwitch units, the HA mode must be active-passive. Starting in FortiOS 6.2.0, the FortiGate HA mode can be either active-passive or active-active.

HA-mode FortiGate units managing a stack of several FortiSwitch units

The master and slave FortiGate units both connect a FortiLink to the first FortiSwitch unit and (optionally) to the last FortiSwitch unit. The FortiLink ports and interface type must match on the two FortiGate units.

When using an aggregate interface for the active/standby FortiLink configuration, make sure the FortiLink split interface is enabled (this forces one link to be active and the rest to be standby links, which avoids loops in the network). This option can be disabled later if you enable an MCLAG. See Transitioning from a FortiLink split interface to a FortiLink MCLAG.

NOTE:

• When you are using the aggregate interface on the FortiGate unit for the FortiLink interface, the lacp-mode of the FortiLink aggregate interface must be set to static. Unless MCLAG is enabled and you are using 6.2.0 or later, see Transitioning from a FortiLink split interface to a FortiLink MCLAG for details.
• Before FortiOS 6.2.0, when using HA-mode FortiGate units to manage FortiSwitch units, the HA mode must be active-passive. Starting in FortiOS 6.2.0, the FortiGate HA mode can be either active-passive or active-active.

HA-mode FortiGate units managing a FortiSwitch two-tier topology

The distribution FortiSwitch unit connects to the master and slave FortiGate units. The FortiLink port(s) and interface type must match on the two FortiGate units.

NOTE: Before FortiOS 6.2.0, when using HA-mode FortiGate units to manage FortiSwitch units, the HA mode must be active-passive. Starting in FortiOS 6.2.0, the FortiGate HA mode can be either active-passive or active-active.

Single FortiGate unit managing multiple FortiSwitch units (using a hardware or software switch interface)

The FortiGate unit connects directly to each FortiSwitch unit. Each of these FortiLink ports is added to the logical hardware-switch or software-switch interface on the FortiGate unit.

Optionally, you can connect other devices to the FortiGate logical interface. These devices, which must support IEEE 802.1q VLAN tagging, will have Layer 2 connectivity with the FortiSwitch ports.

NOTE:

• Using the hardware or software switch interface in FortiLink mode is not recommended in most cases. It can be used when the traffic on the ports is very light because all traffic across the switches moves through the FortiGate unit.
• Do not create loops or rings in this topology.

HA-mode one-tier MCLAG

HA-mode FortiGate units connect to redundant distribution FortiSwitch units. Access FortiSwitch units are arranged in a stack in each IDF, connected to both distribution switches.

For the FortiLink connection to each distribution switch, you create a FortiLink split interface (an aggregate interface that contains one active link and one standby link).

NOTE:

• Before FortiSwitchOS 3.6.4, MCLAG was not supported when access rings were present. Starting with FortiSwitchOS 3.6.4, MCLAG is supported, even with access rings present.
• Before FortiOS 6.2.0, when using HA-mode FortiGate units to manage FortiSwitch units, the HA mode must be active-passive. Starting in FortiOS 6.2.0, the FortiGate HA mode can be either active-passive or active-active.
• When you are using the aggregate interface on the FortiGate unit for the FortiLink interface, the lacp-mode of the FortiLink aggregate interface must be set to static. Unless MCLAG is enabled and you are using 6.2.0 or later, see Transitioning from a FortiLink split interface to a FortiLink MCLAG for details.

• On the global switch level, mclag-stp-aware must be enabled, and STP must be enabled on all ICL trunks.

• This is only an example topology. Other combinations of FortiGate units and FortiSwitch units can be used to create a similar topology.

NOTE: If you are going to use IGMP snooping with an MCLAG topology:

• On the global switch level, mclag-igmp-aware must be enabled,
• The igmps-flood-traffic and igmps-flood-report settings must be disabled on the ISL and FortiLink trunks; but the igmps-flood-traffic and igmps-flood-report settings must be enabled on ICL trunks.
• IGMP proxy must be enabled.

 

Dual-homed servers connected to a pair of FortiSwitch units using an MCLAG

To configure a multichassis LAG, you need to configure FortiSwitch 1 and FortiSwitch 2 as MCLAG peer switches before creating a two-port LAG. Use the set mclag-icl enable command to create an inter-chassis link (ICL) on each FortiSwitch unit (see Transitioning from a FortiLink split interface to a FortiLink MCLAG). Then you set up two MCLAGs towards the servers, each MCLAG using one port from each FortiSwitch unit.

This topology is supported when the FortiGate unit is in HA mode.

NOTE: On the global switch level, mclag-stp-aware must be enabled, and STP must be enabled on all ICL trunks.

NOTE: If you are going to use IGMP snooping with an MCLAG topology:

• On the global switch level, mclag-igmp-aware must be enabled,
• The igmps-flood-traffic and igmps-flood-report settings must be disabled on the ISL and FortiLink trunks; but the igmps-flood-traffic and igmps-flood-report settings must be enabled on ICL trunks.
• IGMP proxy must be enabled.

 

Step 1: Ensure the MCLAG ICL is already configured between FortiSwitch 1 and FortiSwitch 2.

diagnose switch mclag icl

Step 2: Configure a trunk in FortiSwitch 1 and then configure a trunk in FortiSwitch 2.

The trunk names must match.

Step 3: Set up the servers.

To set up Server 1:

config switch trunk

edit server_1

set members port10

set mclag enable

next

edit server_2

set members port15

set mclag enable

next

end

To set up Server 2:

config switch trunk

edit server_1

set members port10

set mclag enable

next

edit server_2

set members port15

set mclag enable

next

end

If you disable the MCLAG ICL (with the set mclag-icl disable command), you need to enable the fortilink-split-interface.

Standalone FortiGate unit with dual-homed FortiSwitch access

This network topology provides high port density with two tiers of FortiSwitch units.

Use the set mclag-icl enable command to create an ICL on each FortiSwitch unit (see Transitioning from a FortiLink split interface to a FortiLink MCLAG).

NOTE: On the global switch level, mclag-stp-aware must be enabled, and STP must be enabled on all ICL trunks.

NOTE: If you are going to use IGMP snooping with an MCLAG topology:

• On the global switch level, mclag-igmp-aware must be enabled,
• The igmps-flood-traffic and igmps-flood-report settings must be disabled on the ISL and FortiLink trunks; but the igmps-flood-traffic and igmps-flood-report settings must be enabled on ICL trunks.
• IGMP proxy must be enabled.

HA-mode FortiGate units with dual-homed FortiSwitch access

In HA mode, only one FortiGate is active at a time. If the active FortiGate unit fails, the backup FortiGate unit becomes active.

Use the set mclag-icl enable command to create an ICL on each FortiSwitch unit (see Transitioning from a FortiLink split interface to a FortiLink MCLAG).

NOTE:

• Before FortiOS 6.2.0, when using HA-mode FortiGate units to manage FortiSwitch units, the HA mode must be active-passive. Starting in FortiOS 6.2.0, the FortiGate HA mode can be either active-passive or active-active.
• On the global switch level, mclag-stp-aware must be enabled, and STP must be enabled on all ICL trunks.

NOTE: If you are going to use IGMP snooping with an MCLAG topology:

• On the global switch level, mclag-igmp-aware must be enabled,
• The igmps-flood-traffic and igmps-flood-report settings must be disabled on the ISL and FortiLink trunks; but the igmps-flood-traffic and igmps-flood-report settings must be enabled on ICL trunks.
• IGMP proxy must be enabled.

Multi-tiered MCLAG with HA-mode FortiGate units

NOTE:

• Before FortiOS 6.2.0, when using HA-mode FortiGate units to manage FortiSwitch units, the HA mode must be active-passive. Starting in FortiOS 6.2.0, the FortiGate HA mode can be either active-passive or active-active.
• In this topology, you must use the auto-isl-port-group setting as described in the following configuration example. This setting instructs the switches to group ports from MCLAG peers together into one MCLAG when the inter-switch link (ISL) is formed.
• The inter-chassis link (ICL) and auto-isl-port-group settings must be done directly on the FortiSwitch unit.

• On the global switch level, mclag-stp-aware must be enabled, and STP must be enabled on all ICL trunks.

• CLI commands in red are manually configured.

NOTE: If you are going to use IGMP snooping with an MCLAG topology:

• On the global switch level, mclag-igmp-aware must be enabled,
• The igmps-flood-traffic and igmps-flood-report settings must be disabled on the ISL and FortiLink trunks; but the igmps-flood-traffic and igmps-flood-report settings must be enabled on ICL trunks.
• IGMP proxy must be enabled.

To configure a multi-tiered MCLAG with HA-mode FortiGate units:

1. Configure FortiSwitch-1 and FortiSwitch-2 for the tier-1 MCLAG:

   For FortiSwitch-1, enable the ICL on the ISL formed with the MCLAG peer switch:

   config switch trunk

   edit "D243Z14000288-0" // trunk name derived from FortiSwitch-2 SN

   set mode lacp-active

   set auto-isl 1

   set mclag-icl enable

   set members "port21" "port22"

   end

   For FortiSwitch-2, enable the ICL on the ISL formed with the MCLAG peer switch:

   config switch trunk

   edit "D243Z14000289-0" // trunk name derived from FortiSwitch-1 SN

   set mode lacp-active

   set auto-isl 1

   set mclag-icl enable

   set members "port21" "port22"

   end

2. Continue to configure FortiSwitch-1 for the tier-1 MCLAG:

   1. Configure the two auto-isl-port-groups based on the topology diagram. The group name must match the name that is configured on the peer switch.

      config switch auto-isl-port-group

      edit "distribute-1"

      set members "port1" "port2"

      next

      edit "distribute-2"

      set members "port3" "port4"

      end

   2. After you complete the CLI commands in Steps 1 and 2a, the trunks are automatically formed:

      config switch trunk

      edit "D243Z14000288-0"

      set mode lacp-active

      set auto-isl 1

      set mclag-icl enable

      set members "port21" "port22"

      next

      edit "FG100D3G15817028" // trunk name derived from FortiGate-1

      set mclag enable

      set members "port24" "port23"

      next

      edit "distribute-1"

      set mode lacp-active

      set auto-isl 1

      set mclag enable

      set members "port1" "port2"

      next

      edit "distribute-2"

      set mode lacp-active

      set auto-isl 1

      set mclag enable

      set members "port3" "port4"

      next

      end

3. Continue to configure FortiSwitch-2 for the tier-1 MCLAG:

   1. Configure the two auto-isl-port-groups based on the topology diagram. The group name must match the name that is configured on the peer switch.

      config switch auto-isl-port-group

      edit "distribute-1"

      set members "port1" "port2"

      next

      edit "distribute-2"

      set members "port3" "port4"

      end

   2. After you complete the CLI commands in Steps 1 and 3a, the trunks are automatically formed:

      config switch trunk

      edit "D243Z14000288-0"

      set mode lacp-active

      set auto-isl 1

      set mclag-icl enable

      set members "port21" "port22"

      next

      edit "FG100D3G15817032" // trunk name derived from FortiGate-2

      set mclag enable

      set members "port24" "port23"

      next

      edit "distribute-1"

      set mode lacp-active

      set auto-isl 1

      set mclag enable

      set members "port1" "port2"

      next

      edit "distribute-2"

      set mode lacp-active

      set auto-isl 1

      set mclag enable

      set members "port3" "port4"

      next

      end

4. Tier-2 MCLAGs. Enable the ICL between the MCLAG peer switches. For example, configure FortiSwitch-6 as follows.

   1. Change the tier-2 MCLAG peer switches to FortiLink mode and connect them to each other. Enable the ICL on the ISL formed with the MCLAG peer switches.

      config switch trunk

      edit "8DN3X15000026-0" // trunk name derived from FortiSwitch-7 SN

      set mode lacp-active

      set auto-isl 1

      set mclag-icl enable

      set members "port43" "port44"

      end

   2. The trunks are automatically formed as below:

      config switch trunk

      edit "8DN3X15000026-0"

      set mode lacp-active

      set auto-isl 1

      set mclag-icl enable

      set members "port43" "port44"

      next

      edit "_FlInK1_MLAG0_"

      set mode lacp-active

      set auto-isl 1

      set mclag enable

      set members "port48" "port47"

      next

      end

5. Access FortiSwitch units. The access switch trunks are formed automatically as below.

   On FortiSwitch-6:

   config switch trunk

   edit "_FlInK1_MLAG0_"

   set mode lacp-active

   set auto-isl 1

   set mclag enable

   set members "port48" "port47"

   next

   end

    

   On FortiSwitch-7:

   config switch trunk

   edit "_FlInK1_MLAG0_"

   set mode lacp-active

   set auto-isl 1

   set mclag enable

   set members "port47" "port48"

   next

   end

   If you disable the MCLAG ICL (with the set mclag-icl disable command), you need to enable the fortilink-split-interface.

Three-tier FortiLink MCLAG configuration

To create a three-tier FortiLink MCLAG topology, use FortiOS 6.2.3 GA or later and FortiSwitchOS 6.2.3 GA or later.

To configure the two FortiGate units:

1. Set up an active-passive HA configuration.
2. (Optional) Disable override in the HA CLI configuration.
3. Use the GUI or CLI to create the FortiLink interface.
4. Configure the FortiLink interface:
   
   

   config system interface

   edit <FortiLink_interface>

   set lacp-mode active

   set fortilink-neighbor-detect lldp

   set fortilink-split-interface disable

   set lldp-reception enable

   set lldp-transmission enable

   next

   end

   
   
5. Change the default auto-config policy:

    

   config switch-controller auto-config policy

   edit default

   set poe-status disable

   next

   end

    

6. Change the firmware upgrade mode:

    

   config switch-controller global

   set https-image-push enable

   end

To configure the FortiSwitch units in the core:

1. Find the trunk between the two MCLAG switches. Enable mclag-icl on the MCLAG-ICL trunk. The default name of the MCLAG-ICL trunk is the last 13 characters of the peer switch name plus “-0”.

    

   config switch trunk

   edit <MCLAG-ICL_trunk_name>

   set mclag-icl enable

   next

   end

    

2. Create downlink trunks on the MCLAG-ICL switches.

   Note: Only the trunks from the higher tier MCLAG-ICL switches to the next tier MCLAG-ICL switches need this configuration.

To configure the three-tier MCLAG topology shown in the following figure:

1. Configure the tier-1 MCLAG switches.
   1. Connect switch 1 and switch 2 to the FortiGate units and interconnect switch 1 and switch 2.
   2. Wait for both switches to change to FortiLink mode and for both FortiLinks to be up.
   3. Configure the ICL trunks on the inter-switch trunks to form MCLAG switches in FortiLink mode.
   4. Use the diagnose switch mclag peer-consistency-check CLI command to verify that the MCLAG-ICL trunk formed successfully.
   5. Add an auto-isl-port-group for the tier-2 MCLAG switches on both switch 1 and switch 2:

       

      config switch auto-isl-port-group

      edit tier2-closet-1

      set members port1

      next

      edit tier2-closet-2

      set members port2

      next

      end

       

2. Wire all switches in closet 1 by following the figure. Do not make the dotted-line connections for now. Wait for all switches to be up in FortiLink mode.
3. Add two auto-isl-port-groups for the tier-3 MCLAG switches on both switch 3 and switch 4:

    

   config switch auto-isl-port-group

   edit tier-2-closet-<1>-downlink-trunk-A

   set member <port_name>

   next

   edit tier-2-closet-<1>-downlink-trunk-B

   set member <port_name>

   next

   end

    

4. Enable the tier-2 MCLAG-ICL trunk on switch 4 using the FortiOS CLI of the switch console port.
5. Enable the tier-3 MCLAG-ICL trunks on switch 6 and switch 8.
   NOTE: The trunk must be configured from the end of the daisy-chain switch.
6. Enable the tier-3 MCLAG-ICL trunks on switch 5 and switch 7.
7. Enable the tier-2 MCLAG-ICL trunk on switch 3.
8. Verify that all the FortiLinks are up and double-check that the MCLAG-ICL configuration on each MCLAG switch.
9. Connect switch 4 to switch 2.
10. Verify that the FortiLinks are up.
11. Connect switch 6 and switch 8 to switch 4.
12. Verify that the FortiLinks are up.
13. Use the diagnose switch mclag peer CLI command to verify that the tier-1, tier-2, and tier-3 MCLAG-switches are formed correctly.
14. Check the traffic on switch 1 and switch 2 during the configuration.
15. Repeat steps 2 to 14 for closet 2.
16. All FortiLinks should be up.

HA-mode FortiGate units using hardware-switch interfaces and STP

In most FortiLink topologies, MCLAG or LAG configurations are used for FortiSwitch redundancy. However, some FortiGate models (such as the FG-60E model) do not support the FortiLink aggregate interface.

The following network topology uses a hardware-switch interface on each FortiGate unit. Each FortiSwitch unit is connected to a single port of the hardware-switch interface of the FortiGate unit. The inter-switch link (ISL) between the FortiSwitch units provides redundancy.

For this network topology to function, use the following commands on each FortiLink hardware-switch interface:

config system interface

edit <FortiLink_hardware_switch_interface>

set stp enable

end

 

NOTE:

• The FortiLink interface uses the Link Layer Discovery Protocol (LLDP) for neighbor detection.
• Spanning Tree Protocol (STP) and STP forwarding are both supported by the FortiLink hardware-switch interface.
• The software-switch interface is not supported.

 

HA-mode FortiGate units in remote sites

There are two sites in this topology, each with a FortiGate unit. The two sites share the FortiGate units in active-passive HA mode. The FortiGate units use the FortiSwitch units in FortiLink mode as the heartbeat connections because of limited physical connections between the two sites.

FortiOS 6.4.2 or higher and FortiSwitchOS 6.4.2 or higher are required.

The following steps are an example of how to configure this topology:

1. Disconnect the physical connections between the two sites.
2. On Site 1:
   1. Use the FortiGate unit to establish the FortiLinks on Site 1. See Connecting FortiLink ports.
   2. Enable the MCLAG-ICL on the core switches of Site 1. See Transitioning from a FortiLink split interface to a FortiLink MCLAG.
   3. Enable the HA mode and set the heartbeat ports on FortiGate-1. FortiGate port1 and port2 are used as HA heartbeat ports in this example. For example, set hbdev "port1" 242 "port2" 25.
   4. Create a switch VLAN or VLANs dedicated to the FortiGate HA heartbeats between the two FortiGate units. For example:

       

      config system interface

      edit "hb1"

      set vdom "vdom name"

      set vlanid 998

      next

      edit "hb2"

      set vdom "vdom name"

      set vlanid 999

      next

      end

       

   5. Under the config switch-controller managed-switch command, set the native VLAN of the switch ports connected to the heartbeat ports using the VLAN created in step 2d.
      
      In this example, you need to assign port1 of core-switch1 to vlan998 and connect port1 of the active FortiGate unit to port1 of core-switch1. Then you need to assign port1 of core-switch2 to vlan999 and connect port2 of the active FortiGate unit to port1 of core-switch2.

       

      config switch-controller managed-switch

      edit <site1-core-switch1>

      edit "port1"

      set vlan "hb1"

      next

      end

      edit <site1-core-switch2>

      edit "port1"

      set vlan "hb2"

      next

      end

       

   6. Make sure all FortiLinks are up.
3. On Site 2:
   1. Configure Site 2 using the same configuration as step 2, except for the HA priority.
   2. Make sure all FortiLinks are up.
4. Disconnect the physical connections for the FortiGate HA and FortiLink interface on Site 2.
5. Connect the cables between the two pairs of core switches in Site 1 and Site 2.
6. On both sites:
   1. On the MCLAG Peer Group switches at Site 1, use the config switch auto-isl-port-group command in the FortiSwitch CLI to group the ports to Site 2. See Multi-tiered MCLAG with HA-mode FortiGate units or Three-tier FortiLink MCLAG configuration.
   2. On the MCLAG Peer Group switches at Site 2 , use the config switch auto-isl-port-group command in the FortiSwitch CLI to group the ports to Site 1. See Multi-tiered MCLAG with HA-mode FortiGate units or Three-tier FortiLink MCLAG configuration.
   3. Make sure all the FortiLinks are up.
7. Connect the FortiGate HA and FortiLink interface connections on Site 2.
8. Check the configuration:
   1. On both sites, enter the get system ha status command on the FortiGate unit to check the HA status.
   2. On the active (master) FortiGate unit, enter the execute switch-controller get-conn-status command to check the FortiLink state.
9. In the GUI, the example configuration looks like the following:
   

   

FortiLink with an HA cluster of four FortiGate units

A FortiGate HA cluster consists of two to four FortiGate units configured for HA operation. Each FortiGate in a cluster is called a cluster unit. All cluster units must be the same FortiGate model with the same FortiOS firmware build installed. All cluster units must also have the same hardware configuration (for example, the same number of hard disk) and be running in the same operating mode (NAT mode or transparent mode).

In addition, the cluster units must be able to communicate with each other through their heartbeat interfaces. This heartbeat communication is required for the cluster to be created and to continue operating. Without it, the cluster acts like a collection of standalone FortiGate units.

On startup, after configuring the cluster units with the same HA configuration and connecting their heartbeat interfaces, the cluster units use the FortiGate Clustering Protocol (FGCP) to find other FortiGate units configured for HA operation and to negotiate to create a cluster. During cluster operation, the FGCP shares communication and synchronization information among the cluster units over the heartbeat interface link. This communication and synchronization is called the FGCP heartbeat or the HA heartbeat. Often, this is shortened to just heartbeat.

NOTE: You can create an FGCP cluster of up to four FortiGate units.

The cluster uses the FGCP to select the primary unit, and to provide device, link, and session failover. The FGCP also manages the two HA modes; active-passive (failover HA) and active-active (load-balancing HA).

The FGCP supports a cluster of two, three, or four FortiGate units. You can add more than two units to a cluster to improve reliability: if two cluster units fail the third will continue to operate and so on. A cluster of three or four units in active-active mode may improve performance because another cluster unit is available for security profile processing. However, active-active FGCP HA results in diminishing performance returns as you add units to the cluster, so the additional performance achieved by adding the third cluster unit might not be worth the cost.

There are no special requirements for clusters of more than two units. Here are a few recommendations though:

• The matching heartbeat interfaces of all of the cluster units must be able to communicate with each other. So each unitʼs matching heartbeat interface should be connected to the same switch. If the ha1 interface is used for heartbeat communication, the ha1 interfaces of all of the units in the cluster must be connected together so communication can happen between all of the cluster units over the ha1 interface.
• Redundant heartbeat interfaces are recommended. You can reduce the number of points of failure by connecting each matching set of heartbeat interfaces to a different switch. This is not a requirement; however, and you can connect both heartbeat interfaces of all cluster units to the same switch. However, if that switch fails the cluster will stop forwarding traffic.
• For any cluster, a dedicated switch for each heartbeat interface is recommended because of the large volume of heartbeat traffic and to keep heartbeat traffic off of other networks, but it is not required.
• Full mesh HA can scale to three or four FortiGate units. Full mesh HA is not required if you have more than two units in a cluster.
• Virtual clustering can only be done with two FortiGate units.

The following network topology uses four FortiGate units; each is a 3200D model and is running FortiOS 6.4.0 build 1533. The FortiSwitch models are 1048E, 448D, and 426EF; they are running FortiSwitchOS 6.2.0 build 0202:

FortiLink over a point-to-point layer-2 network

Starting in FortiSwitchOS 6.4.0, you can run FortiLink mode over a point-to-point layer-2 network. To create this topology, you form an inter-switch link (ISL) between two FortiSwitch units over a layer-2 device or non-FortiSwitch device (such as a wireless bridge) and configure the tag protocol identifier (TPID) between the two FortiSwitch units.

NOTE:

• The set fortilink-p2p-tpid command is not supported on the FS-108E, FS-108E-POE, FS-108E-FPOE, FS-124E, FS-124E-POE, FS-124E-FPOE, FS-148E, and FS-148E-POE models.
• The set fortlink-p2p command is available in Fortilink mode and standalone mode. The set fortilink-p2p-tpid command is available only in FortiLink mode.

 

1. Enable the FortiLink point-to-point network on each FortiSwitch unit:

    

   config switch physical-port

   edit <port_name>

   set fortlink-p2p enable

   end

    

2. Make certain that the FortiLink point-to-point TPID value is the same on each FortiSwitch unit. By default, it is 0x8100.

    

   config switch global

   set fortilink-p2p-tpid <0x0001-0xfffe>

   end

Grouping FortiSwitch units

You can simplify the configuration and management of complex topologies by creating FortiSwitch groups. A group can include one or more FortiSwitch units and you can include different models in a group.

Using the GUI:

1. Go to WiFi & Switch Controller > Managed FortiSwitch.
2. Select Create New > FortiSwitch Group.
3. In the Name field, enter a name for the FortiSwitch group.
4. In the Members field, click + to select which switches to include in the FortiSwitch group.
5. In the Description field, enter a description of the FortiSwitch group.
6. Select OK.

Using the CLI:

config switch-controller switch-group

edit <name>

set description <string>

set members <serial-number> <serial-number> ...

end

end

Grouping FortiSwitch units allows you to restart all of the switches in the group instead of individually. For example, you can use the following command to restart all of the FortiSwitch units in a group named my-sw-group:

execute switch-controller switch-action restart delay switch-group my-sw-group

Upgrading the firmware of FortiSwitch groups is easier, too, because fewer commands are needed. See Firmware upgrade of stacked or tiered FortiSwitch units.

Stacking configuration

To set up stacking:

1. Configure the active FortiLink interface on the FortiGate unit.
2. (Optional) Configure the standby FortiLink interface.
3. Connect the FortiSwitch units together, based on your chosen topology.

1. Configure the active FortiLink

Configure the FortiLink interface (as described in the Using the FortiGate GUI chapter).

When you configure the FortiLink interface, the stacking capability is enabled automatically.

2. Configure the standby FortiLink

Configure the standby FortiLink interface. Depending on your configuration, the standby FortiLink might connect to the same FortiGate unit as the active FortiLink or to a different FortiGate unit.

If the FortiGate unit receives discovery requests from two FortiSwitch units, the link from one FortiSwitch unit will be selected as active, and the link from other FortiSwitch unit will be selected as standby.

If the active FortiLink fails, the FortiGate unit converts the standby FortiLink to active.

3. Connect the FortiSwitch units

Refer to the topology diagrams to see how to connect the FortiSwitch units.

Inter-switch links (ISLs) form automatically between the stacked switches.

The FortiGate unit will discover and authorize all of the FortiSwitch units that are connected. After this, the FortiGate unit is ready to manage all of the authorized FortiSwitch units.

Disable stacking

To disable stacking, execute the following commands from the FortiGate CLI. In the following example, port4 is the FortiLink interface:

config system interface

edit port4

set fortilink-stacking disable

end

end

Firmware upgrade of stacked or tiered FortiSwitch units

In this topology, the core FortiSwitch units are model FS-224E, and the access FortiSwitch units are model FS-108E-FPOE. Because the switches are stacked or tiered, the procedure to update the firmware is simpler. The FortiGate unit is running FOS 6.2.2 GA. In the following procedure, the four FortiSwitch units are upgraded from 6.2.1 to 6.2.2.

To upgrade the firmware of stacked or tiered FortiSwitch units:

1. Check that all of the FortiSwitch units are connected and which firmware versions they are running. For example:

   FGT81ETK19001274 # execute switch-controller get-conn-status 
   Managed-devices in current vdom root:
   
   STACK-NAME: FortiSwitch-Stack-flink
   SWITCH-ID         VERSION           STATUS         FLAG   ADDRESS       JOIN-TIME      NAME 
   S108EF5918003577  v6.2.1 (176)      Authorized/Up   -   10.105.22.6     Thu Oct 24 10:47:27 2019    -  
   S108EP5918008265  v6.2.1 (176)      Authorized/Up   -   10.105.22.5     Thu Oct 24 10:47:20 2019    -     
   S224ENTF18001408  v6.2.1 (176)      Authorized/Up   -   10.105.22.2     Thu Oct 24 10:44:36 2019    -    
   S224ENTF18001432  v6.2.1 (176)      Authorized/Up   -   10.105.22.3     Thu Oct 24 10:44:49 2019    -    
   
   Flags: C=config sync, U=upgrading, S=staged, D=delayed reboot pending, E=configuration sync error
   Managed-Switches: 4 (UP: 4 DOWN: 0)

2. (Optional) To speed up how fast the image is pushed from the FortiGate unit to the FortiSwitch units, enable the HTTPS image push instead of the CAPWAP image push. For example:

   FGT81ETK19001274 # config switch-controller global 
   FGT81ETK19001274 (global) # set https-image-push enable 
   FGT81ETK19001274 (global) # end

3. Download the file for the FortiSwitchOS 6.2.2 GA build 194 in the FortiGate unit. For example:

   FGT81ETK19001274 # execute switch-controller switch-software upload tftp FSW_224E-v6-build0194-FORTINET.out 10.105.16.15
   
   Downloading file FSW_224E-v6-build0194-FORTINET.out from tftp server 10.105.16.15...
   #########################
   Image checking ...
   Image MD5 calculating ...
   Image Saving S224EN-IMG.swtp ...
   Successful!
   
   File Syncing...
   
   FGT81ETK19001274 # execute switch-controller switch-software upload tftp FSW_108E_POE-v6-build0194-FORTINET.out 10.105.16.15
   
   Downloading file FSW_108E_POE-v6-build0194-FORTINET.out from tftp server 10.105.16.15...
   ##################
   Image checking ...
   Image MD5 calculating ...
   Image Saving S108EP-IMG.swtp ...
   Successful!
   
   File Syncing...
   
   FGT81ETK19001274 # execute switch-controller switch-software upload tftp FSW_108E_FPOE-v6-build0194-FORTINET.out 10.105.16.15
   
   Downloading file FSW_108E_FPOE-v6-build0194-FORTINET.out from tftp server 10.105.16.15...
   ##################
   Image checking ...
   Image MD5 calculating ...
   Image Saving S108EF-IMG.swtp ...
   Successful!
   
   File Syncing...
   
   FGT81ETK19001274 #

4. Check the downloaded FortiSwitch image. For example:

   FGT81ETK19001274 # execute switch-controller switch-software list-available 
   
   ImageName              ImageSize(B)   ImageInfo               Uploaded Time  
   S108EF-IMG.swtp        19574769       S108EF-v6.2-build194    Thu Oct 24 13:03:51 2019
   S108EP-IMG.swtp        19583362       S108EP-v6.2-build194    Thu Oct 24 13:03:23 2019
   S224EN-IMG.swtp        27159659       S224EN-v6.2-build194    Thu Oct 24 13:03:02 2019
   
   FGT81ETK19001274 #

5. Start the image staging. For example:

   FGT81ETK19001274 #  execute switch-controller switch-software stage all S224EN-IMG.swtp
   Staged Image Version S224EN-v6.2-build194
   Image staging operation is started for FortiSwitch S224ENTF18001408 ...
   Image staging operation is started for FortiSwitch S224ENTF18001432 ...
   
   FGT81ETK19001274 # execute switch-controller switch-software stage all S108EF-IMG.swtp
   Staged Image Version S108EF-v6.2-build194
   Image staging operation is started for FortiSwitch S108EF5918003577 ...
   
   FGT81ETK19001274 # execute switch-controller switch-software stage all S108EP-IMG.swtp
   Staged Image Version S108EP-v6.2-build194
   Image staging operation is started for FortiSwitch S108EP5918008265 ...

6. Check the status of the image staging. For example:

   FGT81ETK19001274 # execute switch-controller get-upgrade-status
   Device    Running-version                                Status      Next-boot
   ===========================================================================================
   VDOM : root
   S224ENTF18001408  S224EN-v6.2.1-build176,190620 (GA)             (100/0/0)   S224EN-v6.2-build176       (Staging) 
   S224ENTF18001432  S224EN-v6.2.1-build176,190620 (GA)             (100/0/0)   S224EN-v6.2-build176       (Staging) 
   S108EP5918008265  S108EP-v6.2.1-build176,190620 (GA)             (18/0/0)   S108EP-v6.2-build176        (Staging) 
   S108EF5918003577  S108EF-v6.2.1-build176,190620 (GA)             (25/0/0)   S108EF-v6.2-build176        (Staging)

7. Verify that the image staging has completed. For example:

   FGT81ETK19001274 # execute switch-controller get-upgrade-status
   Device    Running-version                                Status      Next-boot
   ===========================================================================================
   VDOM : root
   S224ENTF18001408  S224EN-v6.2.1-build176,190620 (GA)             (0/100/100)   S224EN-v6.2-build194     (Idle) 
   S224ENTF18001432  S224EN-v6.2.1-build176,190620 (GA)             (0/100/100)   S224EN-v6.2-build194     (Idle) 
   S108EP5918008265  S108EP-v6.2.1-build176,190620 (GA)             (0/100/100)   S108EP-v6.2-build194     (Idle) 
   S108EF5918003577  S108EF-v6.2.1-build176,190620 (GA)             (0/100/100)   S108EF-v6.2-build194     (Idle)

8. Reboot all switches (or reboot the switches by group). For example:

   FGT81ETK19001274 # execute switch-controller switch-action restart delay all
   Delayed restart operation is requested for FortiSwitch S224ENTF18001408 ...
   Delayed restart operation is requested for FortiSwitch S224ENTF18001432 ...
   Delayed restart operation is requested for FortiSwitch S108EP5918008265 ...
   Delayed restart operation is requested for FortiSwitch S108EF5918003577 ...

9. Check the status of the switch reboot. For example:

   FGT81ETK19001274 # execute switch-controller switch-action restart delay all
   Delayed restart operation is requested for FortiSwitch S224ENTF18001408 ...
   Delayed restart operation is requested for FortiSwitch S224ENTF18001432 ...
   Delayed restart operation is requested for FortiSwitch S108EP5918008265 ...
   Delayed restart operation is requested for FortiSwitch S108EF5918003577 ...
   
   FGT81ETK19001274 # execute switch-controller get-upgrade-status
   Device    Running-version                                Status      Next-boot
   ===========================================================================================
   VDOM : root
   S224ENTF18001408                        Prepping for delayed restart triggered ... please wait for switch to reboot in a moment
   S224ENTF18001432                        Prepping for delayed restart triggered ... please wait for switch to reboot in a moment
   S108EP5918008265                        Prepping for delayed restart triggered ... please wait for switch to reboot in a moment
   S108EF5918003577                        Prepping for delayed restart triggered ... please wait for switch to reboot in a moment
   
   FGT81ETK19001274 # execute switch-controller get-conn-status 
   Managed-devices in current vdom root:
   
   STACK-NAME: FortiSwitch-Stack-flink
   SWITCH-ID         VERSION           STATUS         FLAG   ADDRESS      JOIN-TIME       NAME 
   S108EF5918003577  v6.2.1 ()         Authorized/Down D   0.0.0.0         N/A               -    
   S108EP5918008265  v6.2.1 ()         Authorized/Down D   0.0.0.0         N/A               -     
   S224ENTF18001408  v6.2.1 ()         Authorized/Down D   0.0.0.0         N/A               -    
   S224ENTF18001432  v6.2.1 ()         Authorized/Down D   0.0.0.0         N/A               -    
   
   Flags: C=config sync, U=upgrading, S=staged, D=delayed reboot pending, E=configuration sync error
   Managed-Switches: 4 (UP: 0 DOWN: 4)
   
   FGT81ETK19001274 # 

10. Wait for a while before checking that all switches are online. For example:

    FGT81ETK19001274 # execute switch-controller get-upgrade-status
    Device    Running-version                                Status      Next-boot
    ===========================================================================================
    VDOM : root
    S224ENTF18001408  S224EN-v6.2.2-build194,191018 (GA)             (0/100/100)   S224EN-v6.2-build194     (Idle) 
    S224ENTF18001432  S224EN-v6.2.2-build194,191018 (GA)             (0/100/100)   S224EN-v6.2-build194     (Idle) 
    S108EP5918008265  S108EP-v6.2.2-build194,191018 (GA)             (0/100/100)   S108EP-v6.2-build194     (Idle) 
    S108EF5918003577  S108EF-v6.2.2-build194,191018 (GA)             (0/100/100)   S108EF-v6.2-build194     (Idle) 
    
    FGT81ETK19001274 # execute switch-controller get-conn-status   
    Managed-devices in current vdom root:
    
    STACK-NAME: FortiSwitch-Stack-flink
    SWITCH-ID         VERSION           STATUS         FLAG   ADDRESS              JOIN-TIME            NAME            
    S108EF5918003577  v6.2.2 (194)      Authorized/Up   -   10.105.22.6     Thu Oct 24 13:22:27 2019    -     
    S108EP5918008265  v6.2.2 (194)      Authorized/Up   -   10.105.22.5     Thu Oct 24 13:22:41 2019    -     
    S224ENTF18001408  v6.2.2 (194)      Authorized/Up   -   10.105.22.2     Thu Oct 24 13:20:11 2019    -    
    S224ENTF18001432  v6.2.2 (194)      Authorized/Up   -   10.105.22.3     Thu Oct 24 13:19:58 2019    -    
    
    Flags: C=config sync, U=upgrading, S=staged, D=delayed reboot pending, E=configuration sync error
    Managed-Switches: 4 (UP: 4 DOWN: 0)
    
    FGT81ETK19001274 #

Transitioning from a FortiLink split interface to a FortiLink MCLAG

You can use the FortiLink split interface to connect the FortiLink aggregate interface from one FortiGate unit to two FortiSwitch units. When the FortiLink split interface is enabled, only one link remains active.

In this topology, the FortiLink split interface connects a FortiLink aggregate interface from one FortiGate unit to two FortiSwitch units. The aggregate interface of the FortiGate unit for this configuration contains at least one physical port connected to each FortiSwitch unit.

NOTE:

• Make sure that the split interface is enabled.
• This procedure also applies to a FortiGate unit in HA mode.
• More links can be added between the FortiGate unit and FortiSwitch unit.
• On the global switch level, mclag-stp-aware must be enabled, and STP must be enabled on all ICL trunks.

NOTE: If you are going to use IGMP snooping with an MCLAG topology:

• On the global switch level, mclag-igmp-aware must be enabled,
• The igmps-flood-traffic and igmps-flood-report settings must be disabled on the ISL and FortiLink trunks; but the igmps-flood-traffic and igmps-flood-report settings must be enabled on ICL trunks.
• IGMP proxy must be enabled.

The following procedure uses zero-touch provisioning to change the configuration of the FortiSwitch units without losing their management from the FortiGate unit. The MCLAG-ICL can also be enabled directly using console cables or management ports.

1. Log into FortiSwitch 2 using the Connect to CLI button in the FortiGate GUI, use the get switch lldp auto-isl-status command to find out the name of the trunk connecting the peer switches, and change the ISL to an ICL. For example:

   get switch lldp auto-isl-status

    

   config switch trunk

   edit <trunk_name>

   set

   mclag-icl enable

   next

   end

2. Log into FortiSwitch 1 using the Connect to CLI button in the FortiGate GUI, use the get switch lldp auto-isl-status command to find out the name of the trunk connecting the peer switches, and change the ISL to an ICL. For example:

   get switch lldp auto-isl-status

    

   config switch trunk

   edit <trunk_name>

   set mclag-icl enable

   next

   end

3. Log into the FortiGate unit and disable the split interface. For example:

   config system interface

   edit <aggregate_name>

   set fortilink-split-interface disable

   next

   end

4. From the FortiGate unit, enable the LACP static mode:

   config system interface

   edit <aggregate_name>

   set lacp-mode static

   next

   end

   NOTE: If you are using FortiOS 6.2 or later, use the set lacp-mode active command instead.

5. Check that the LAG is working correctly. For example:

   diagnose netlink aggregate name <aggregate_name>

If you disable the MCLAG ICL (with the set mclag-icl disable command), you need to enable the fortilink-split-interface.