Enabling SR-IOV and creating the VF
Setting up SR-IOV on KVM involves creating a PF for each physical network card in the hardware platform. VFs allow FortiGate-VMs to communicate through the physical network card. VFs are actual PCIe hadware resources and only a limited number of VFs are available for each PF. Each QAT addon card creates three PFs with a maximum capacity of 16 VFs each. Ensure that the VM and QAT card are on the same NUMA node.
To configure SR-IOV and create VFs, see SR-IOV support for virtual networking.
After enabling SR-IOV and setting the VF numbers, review the QAT and NIC VFs:
[root@localhost net]# lshw -c processor -businfo
Bus info Device Class Description
========================================================
cpu@0 processor Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz
cpu@1 processor Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz
pci@0000:b1:00.0 processor C62x Chipset QuickAssist Technology
pci@0000:b1:01.0 processor Intel Corporation
pci@0000:b1:01.1 processor Intel Corporation
pci@0000:b1:01.2 processor Intel Corporation
pci@0000:b1:01.3 processor Intel Corporation
pci@0000:b1:01.4 processor Intel Corporation
pci@0000:b1:01.5 processor Intel Corporation
pci@0000:b1:01.6 processor Intel Corporation
pci@0000:b1:01.7 processor Intel Corporation
pci@0000:b1:02.0 processor Intel Corporation
pci@0000:b1:02.1 processor Intel Corporation
pci@0000:b1:02.2 processor Intel Corporation
pci@0000:b1:02.3 processor Intel Corporation
pci@0000:b1:02.4 processor Intel Corporation
pci@0000:b1:02.5 processor Intel Corporation
pci@0000:b1:02.6 processor Intel Corporation
pci@0000:b1:02.7 processor Intel Corporation
pci@0000:b3:00.0 processor C62x Chipset QuickAssist Technology
pci@0000:b3:01.0 processor Intel Corporation
pci@0000:b3:01.1 processor Intel Corporation
pci@0000:b3:01.2 processor Intel Corporation
pci@0000:b3:01.3 processor Intel Corporation
pci@0000:b3:01.4 processor Intel Corporation
pci@0000:b3:01.5 processor Intel Corporation
pci@0000:b3:01.6 processor Intel Corporation
pci@0000:b3:01.7 processor Intel Corporation
pci@0000:b3:02.0 processor Intel Corporation
pci@0000:b3:02.1 processor Intel Corporation
pci@0000:b3:02.2 processor Intel Corporation
pci@0000:b3:02.3 processor Intel Corporation
pci@0000:b3:02.4 processor Intel Corporation
pci@0000:b3:02.5 processor Intel Corporation
pci@0000:b3:02.6 processor Intel Corporation
pci@0000:b3:02.7 processor Intel Corporation
pci@0000:b5:00.0 processor C62x Chipset QuickAssist Technology
pci@0000:b5:01.0 processor Intel Corporation
pci@0000:b5:01.1 processor Intel Corporation
pci@0000:b5:01.2 processor Intel Corporation
pci@0000:b5:01.3 processor Intel Corporation
pci@0000:b5:01.4 processor Intel Corporation
pci@0000:b5:01.5 processor Intel Corporation
pci@0000:b5:01.6 processor Intel Corporation
pci@0000:b5:01.7 processor Intel Corporation
pci@0000:b5:02.0 processor Intel Corporation
pci@0000:b5:02.1 processor Intel Corporation
pci@0000:b5:02.2 processor Intel Corporation
pci@0000:b5:02.3 processor Intel Corporation
pci@0000:b5:02.4 processor Intel Corporation
pci@0000:b5:02.5 processor Intel Corporation
pci@0000:b5:02.6 processor Intel Corporation
pci@0000:b5:02.7 processor Intel Corporation
The above example has one physical QAT card with three QAT accelerators:
[root@localhost ~]# lspci | grep Ethernet
86:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
86:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
86:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
86:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
88:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
88:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
88:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)
[root@localhost ~]# lshw -c network -businfo | grep X710
pci@0000:86:00.0 p5p1 network Ethernet Controller X710 for 10GbE SFP+
pci@0000:86:00.1 p5p2 network Ethernet Controller X710 for 10GbE SFP+
pci@0000:88:00.0 p7p1 network Ethernet Controller X710 for 10GbE SFP+
pci@0000:88:00.1 p7p2 network Ethernet Controller X710 for 10GbE SFP+