FortiGate 3700F and 3701F fast path architecture
The FortiGate 3700F and 3701F each include three NP7 processors. Front panel data interfaces 5 to 26 and the NP7 processors connect to the integrated switch fabric (ISF). All data traffic passes from the data interfaces through the ISF to the NP7 processors. Data traffic processed by the CPU takes a dedicated data path through the ISF and an NP7 processor to the CPU.
Front panel data interfaces 1 to 4 are connected directly to NP#0 (using the NP7 interface named NP#0-link1) instead of the ISF. Since the ISF introduces latency, interfaces 1 to 4 are ultra low latency interfaces (ULL), and NP7 traffic entering and exiting the FortiGate through these interfaces experiences lower latency than if it were passing through interfaces that are connected to the ISF. To achieve low latency, traffic must enter and exit the FortiGate through the 1 to 4 interfaces.
All supported traffic passing between any two data interfaces can be offloaded by the NP7 processors. This includes traffic passing between an interface connected to the ISF and a ULL interface. If traffic enters or exits through an interface connected to the ISF, it is subject to the latency resulting from passing through the ISF.
The FortiGate 3700F and 3701F models feature the following front panel interfaces:
- Two 10G/5G/2.5G/1G/100M BASE-T RJ45 (MGMT1 and MGMT2, not connected to the NP7 processors).
- Twenty 50/25/10/1 GigE SFP56 (HA1, HA2, 5 to 22) the HA interfaces are not connected to the NP7 processors).
- Four 25/10 GigE SFP28/SFP+ (1 to 4) ultra low latency (ULL), all ULL interfaces operate at the same speed. ULL interfaces bypass the integrated switch fabric (ISF).
- Four 400/200/100/40 GigE QSFP-DD (23 to 26). Each of these interfaces can be split into eight 50GigE interfaces, four 100GigE interfaces, or two 200 GigE interfaces.
The MGMT interfaces are not connected to the NP7 processors. Management traffic passes to the CPU over a dedicated management path that is separate from the data path. You can also dedicate separate CPU resources for management traffic to further isolate management processing from data processing (see Improving GUI and CLI responsiveness (dedicated management CPU)).
The HA interfaces are also not connected to the NP7 processors. To help provide better HA stability and resiliency, HA traffic uses a dedicated physical control path that provides HA control traffic separation from data traffic processing.
The separation of management and HA traffic from data traffic keeps management and HA traffic from affecting the stability and performance of data traffic processing.
You can use the following command to display the FortiGate 3700F and 3701F NP7 configuration. The command output shows that the port1 to port4 interfaces are connected to NP#0. The command output also shows that the port5 to port26 interfaces are connected to all three NP7s.
diagnose npu np7 port-list Front Panel Port: Name Max_speed(Mbps) Dflt_speed(Mbps) Sw_Trunk_Id Sw_Tcam_Id Group_from_vdom Switch_id SW_port_id SW_port_name -------- --------------- --------------- --------------- ---------- --------------- --------- ---------- ------------ port1 25000 10000 8 0 0 n/a n/a n/a port2 25000 10000 8 0 0 n/a n/a n/a port3 25000 10000 8 0 0 n/a n/a n/a port4 25000 10000 8 0 0 n/a n/a n/a port5 50000 10000 8 1 0 0 73 n/a port6 50000 10000 8 2 0 0 72 n/a port7 50000 10000 8 3 0 0 77 n/a port8 50000 10000 8 4 0 0 76 n/a port9 50000 10000 8 5 0 0 79 n/a port10 50000 10000 8 6 0 0 78 n/a port11 50000 10000 8 7 0 0 81 n/a port12 50000 10000 8 8 0 0 80 n/a port13 50000 10000 8 9 0 0 83 n/a port14 50000 10000 8 10 0 0 82 n/a port15 50000 10000 8 11 0 0 1 n/a port16 50000 10000 8 12 0 0 0 n/a port17 50000 10000 8 13 0 0 3 n/a port18 50000 10000 8 14 0 0 2 n/a port19 50000 10000 8 15 0 0 5 n/a port20 50000 10000 8 16 0 0 4 n/a port21 50000 10000 8 17 0 0 7 n/a port22 50000 10000 8 18 0 0 6 n/a port23 400000 400000 8 19 0 0 8 n/a port24 400000 400000 8 20 0 0 16 n/a port25 400000 400000 8 21 0 0 24 n/a port26 400000 400000 8 22 0 0 32 n/a -------- --------------- --------------- --------------- ---------- --------------- --------- ---------- ------------ Name sw_id hash nr_link valid default sw_tid -------- --------------------------------------- -------- --------------------------------------- NP Port: Name Switch_id SW_port_id SW_port_name ------ --------- ---------- ------------ np0_0 0 68 n/a np1_0 0 64 n/a np1_1 0 56 n/a np2_0 0 48 n/a np2_1 0 52 n/a ------ --------- ---------- ------------ * Max_speed: Maximum speed, Dflt_speed: Default speed * SW_port_id: Switch port ID, SW_port_name: Switch port name
The command output also shows the maximum and default speeds of each interface.
The integrated switch fabric distributes sessions from the data interfaces to the NP7 processors. The three NP7 processors have a bandwidth capacity of 200Gigabit x 3 = 600 Gigabit. If all interfaces were operating at their maximum bandwidth, the NP7 processors would not be able to offload all the traffic. You can use NPU port mapping to control how sessions are distributed to NP7 processors.
You can add LAGs to improve performance. For details, see Increasing NP7 offloading capacity using link aggregation groups (LAGs).
The FortiGate-3700F and 3701F cannot be licensed for hyperscale firewall support.
Since the FortiGate-3700F and 3701F have three NP7 processors, the following options are available to configure how the internal switch fabric (ISF) distributes sessions to the NP7 processors:
config system global
config system npu
set hash-config {src-dst-ip | src-ip}
end
For more information, see hash-config {src-dst-ip | 5-tuple | src-ip}.
Splitting the port23 to port26 interfaces
You can use the following command to split each FortiGate 3700F and 3701F 23 to 26 (port23 to port26) GigE QSFP-DD interface.
config system global
config split-port-mode
edit port21
set split-mode {disable | 8x50G | 4x100G | 2x200G}
end
disable restore a split interface to the default (not split) configuration.
8x50G split the interface into eight 50GigE interfaces.
4x100G split the interface into four 100GigE interfaces.
2x200G split the interface into two 200 GigE interfaces.
After splitting one or more interfaces, the FortiGate 3700F and 3701F restarts and when it starts up the split interfaces are available.
|
A configuration change that causes a FortiGate to restart can disrupt the operation of an FGCP cluster. If possible, you should make this configuration change to the individual FortiGates before setting up the cluster. If the cluster is already operating, you should temporarily remove the secondary FortiGate(s) from the cluster, change the configuration of the individual FortiGates and then re-form the cluster. You can remove FortiGate(s) from a cluster using the Remove Device from HA cluster button on the System > HA GUI page. For more information, see Disconnecting a FortiGate. |
For example, use the following command to split the port24 interface into eight 50GigE interfaces:
config system global
config split-port-mode
edit port24
set split-mode 8x50G
end
The FortiGate 3700F and 3701F restarts and when it starts up the port24 interface has been replaced by eight 50 GigE interfaces named port24/1 to port24/8.
By default, the speed of each split interface is set to 50000full (50GigE). These interfaces can operate as 25GigE, 10GigE, or 1GigE interfaces depending on the transceivers and breakout cables. You can use the config system interface command to change the speeds of the split interfaces.
Changing the speed of the 1 to 4 ULL interfaces
By default, the FortiGate-3700F and 3701F front panel ULL data interfaces 1 to 4 operate as 10G SFP+ interfaces. You can use the following command to configure them to operate as 25G SFP28 interfaces:
config system npu
set ull-port-mode 25G
end
Entering this command restarts the FortiGate, so the speed of the ULL interfaces should be changed during a maintenance window. This command changes the speeds of all of the ULL interfaces. All of the ULL interfaces operate at the same speed.
|
|
A configuration change that causes a FortiGate to restart can disrupt the operation of an FGCP cluster. If possible, you should make this configuration change to the individual FortiGates before setting up the cluster. If the cluster is already operating, you should temporarily remove the secondary FortiGate(s) from the cluster, change the configuration of the individual FortiGates and then re-form the cluster. You can remove FortiGate(s) from a cluster using the Remove Device from HA cluster button on the System > HA GUI page. For more information, see Disconnecting a FortiGate. |
You can use the following command to change the ULL interfaces back to the default setting as 10G SFP+ interfaces:
config system npu
set ull-port-mode 10G
end
Entering this command also restarts the FortiGate.
When the speed of the ULL interfaces is set to 25G, the output of the diagnose npu np7 port-list command changes to the following:
diagnose npu np7 port-list Front Panel Port: Name Max_speed(Mbps) Dflt_speed(Mbps) Sw_Trunk_Id Sw_Tcam_Id Group_from_vdom Switch_id SW_port_id SW_port_name -------- --------------- --------------- --------------- ---------- --------------- --------- ---------- ------------ port1 25000 25000 8 0 0 n/a n/a n/a port2 25000 25000 8 0 0 n/a n/a n/a port3 25000 25000 8 0 0 n/a n/a n/a port4 25000 25000 8 0 0 n/a n/a n/a port5 50000 10000 8 1 0 0 73 n/a port6 50000 10000 8 2 0 0 72 n/a port7 50000 10000 8 3 0 0 77 n/a port8 50000 10000 8 4 0 0 76 n/a port9 50000 10000 8 5 0 0 79 n/a port10 50000 10000 8 6 0 0 78 n/a port11 50000 10000 8 7 0 0 81 n/a port12 50000 10000 8 8 0 0 80 n/a port13 50000 10000 8 9 0 0 83 n/a port14 50000 10000 8 10 0 0 82 n/a port15 50000 10000 8 11 0 0 1 n/a port16 50000 10000 8 12 0 0 0 n/a port17 50000 10000 8 13 0 0 3 n/a port18 50000 10000 8 14 0 0 2 n/a port19 50000 10000 8 15 0 0 5 n/a port20 50000 10000 8 16 0 0 4 n/a port21 50000 10000 8 17 0 0 7 n/a port22 50000 10000 8 18 0 0 6 n/a port23 400000 400000 8 19 0 0 8 n/a port24 400000 400000 8 20 0 0 16 n/a port25 400000 400000 8 21 0 0 24 n/a port26 400000 400000 8 22 0 0 32 n/a -------- --------------- --------------- --------------- ---------- --------------- --------- ---------- ------------ Name sw_id hash nr_link valid default sw_tid -------- --------------------------------------- -------- --------------------------------------- NP Port: Name Switch_id SW_port_id SW_port_name ------ --------- ---------- ------------ np0_0 0 68 n/a np1_0 0 64 n/a np1_1 0 56 n/a np2_0 0 48 n/a np2_1 0 52 n/a ------ --------- ---------- ------------ * Max_speed: Maximum speed, Dflt_speed: Default speed * SW_port_id: Switch port ID, SW_port_name: Switch port name
Configuring FortiGate 3700F and 3701F NPU port mapping
The default FortiGate-3700F and 3701F port mapping configuration results in sessions passing from front panel data interfaces to the integrated switch fabric. The integrated switch fabric distributes these sessions among the NP7 processors. Each NP7 processor is connected to the switch fabric with a LAG that consists of two 100-Gigabitinterfaces. The integrated switch fabric distributes sessions to the LAGs and each LAG distributes sessions between the two interfaces connected to the NP7 processor.
You can use NPU port mapping to override how data network interface sessions are distributed to NP7 processors. For example, you can set up NPU port mapping to send all traffic from a front panel data interface or LAG to a specific NP7 processor or group of NP7 processors, or a single NP7 link.
|
|
On the FortiGate 3700F and 3701F you can configure ISF load balancing to change the algorithm that the ISF uses to distribute data interface sessions to NP7 processors. ISF load balancing is configured for an interface, and distributes sessions from that interface to all NP7 processor LAGs. If you have configured NPU port mapping, ISF load balancing distributes sessions from the interface to the NP7 processors and links in the NPU port mapping configuration for that interface. See Configuring ISF load balancing. |
Use the following command to configure FortiGate-3700F and 3701F NPU port mapping:
config system npu-post
config port-npu-map
edit <interface-name>
set npu-group {All-NP | NP0 | NP1 | NP2 | NP0-to-NP1 | NP1-to-NP2 | NP0-link0 | NP0-link1 | NP1-link0 | NP1-link1 | NP2-link0 | NP2-link1} ...
end
end
end
<interface-name> can be a physical interface or a LAG.
|
|
You cannot configure FortiGate-3700F or 3701F port mapping to use the NP0-link1 interface because this interface is used for ULL connections to front panel interfaces 1 to 4. |
All-NP, (the default) distribute sessions among all three NP7 LAGs.
NP0, distribute sessions to the LAG connected to NP0.
NP1, distribute sessions to the LAG connected to NP1.
NP2, distribute sessions to the LAG connected to NP2.
NP0-to-NP1, distribute sessions between the LAG connected to NP0 and the LAG connected to NP1.
NP1-to-NP2, distribute sessions between the LAG connected to NP1 and the LAG connected to NP2.
NP0-link0, send sessions to NP0 link 0.
NP1-link0, send sessions to NP1 link 0.
NP1-link1, send sessions to NP1 link 1.
NP2-link0, send sessions to NP2 link 0.
NP2-link1, send sessions to NP2 link 1.
You can add multiple group names to map traffic to multiple groups of NP7 processors and NP7 processor links. For example, use the following command to distribute sessions from port22 to NP0, NP1, and NP2-link1:
config system npu-post
config port-npu-map
edit port22
set npu-group NP0 NP1 NP2-link1
end
end
end
Group names can't overlap, for example you can't map an interface to both NP1 and NP1-link1.
For example, use the following syntax to assign the FortiGate-3700F port23 and port24 interfaces to NP0 and NP1 and port25 and port26 to NP2:
config system npu-post
config port-npu-map
edit port23
set npu-group NP0 NP1
next
edit port24
set npu-group NP0 NP1
next
edit port25
set npu-group NP2
next
edit port26
set npu-group NP2
end
end
While the FortiGate-3700F or 3701F is processing traffic, you can use the diagnose npu np7 cgmac-stats <npu-id> command to show how traffic is distributed to the NP7 links.
You can use the diagnose npu np7 port-list command to see the current NPU port map configuration. For example, after making the changes described in the example, the output of the diagnose npu np7 port-list command shows different Sw_Trunk_Ids for port23 to port26 and these interfaces are listed in a port mapping summary at the bottom of the command output: