Fortinet Document Library

Version:


Table of Contents

KVM Cookbook

Resources

Upgrade Path Tool
6.2.0
Download PDF
Copy Link

Enabling SR-IOV and creating the VF

Setting up SR-IOV on KVM involves creating a PF for each physical network card in the hardware platform. VFs allow FortiGate-VMs to communicate through the physical network card. VFs are actual PCIe hadware resources and only a limited number of VFs are available for each PF. Each QAT addon card creates three PFs with a maximum capacity of 16 VFs each. Ensure that the VM and QAT card are on the same NUMA node.

To configure SR-IOV and create VFs, see SR-IOV support for virtual networking.

After enabling SR-IOV and setting the VF numbers, review the QAT and NIC VFs:

[root@localhost net]# lshw -c processor -businfo

Bus info Device Class Description

========================================================

cpu@0 processor Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz

cpu@1 processor Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz

pci@0000:b1:00.0 processor C62x Chipset QuickAssist Technology

pci@0000:b1:01.0 processor Intel Corporation

pci@0000:b1:01.1 processor Intel Corporation

pci@0000:b1:01.2 processor Intel Corporation

pci@0000:b1:01.3 processor Intel Corporation

pci@0000:b1:01.4 processor Intel Corporation

pci@0000:b1:01.5 processor Intel Corporation

pci@0000:b1:01.6 processor Intel Corporation

pci@0000:b1:01.7 processor Intel Corporation

pci@0000:b1:02.0 processor Intel Corporation

pci@0000:b1:02.1 processor Intel Corporation

pci@0000:b1:02.2 processor Intel Corporation

pci@0000:b1:02.3 processor Intel Corporation

pci@0000:b1:02.4 processor Intel Corporation

pci@0000:b1:02.5 processor Intel Corporation

pci@0000:b1:02.6 processor Intel Corporation

pci@0000:b1:02.7 processor Intel Corporation

pci@0000:b3:00.0 processor C62x Chipset QuickAssist Technology

pci@0000:b3:01.0 processor Intel Corporation

pci@0000:b3:01.1 processor Intel Corporation

pci@0000:b3:01.2 processor Intel Corporation

pci@0000:b3:01.3 processor Intel Corporation

pci@0000:b3:01.4 processor Intel Corporation

pci@0000:b3:01.5 processor Intel Corporation

pci@0000:b3:01.6 processor Intel Corporation

pci@0000:b3:01.7 processor Intel Corporation

pci@0000:b3:02.0 processor Intel Corporation

pci@0000:b3:02.1 processor Intel Corporation

pci@0000:b3:02.2 processor Intel Corporation

pci@0000:b3:02.3 processor Intel Corporation

pci@0000:b3:02.4 processor Intel Corporation

pci@0000:b3:02.5 processor Intel Corporation

pci@0000:b3:02.6 processor Intel Corporation

pci@0000:b3:02.7 processor Intel Corporation

pci@0000:b5:00.0 processor C62x Chipset QuickAssist Technology

pci@0000:b5:01.0 processor Intel Corporation

pci@0000:b5:01.1 processor Intel Corporation

pci@0000:b5:01.2 processor Intel Corporation

pci@0000:b5:01.3 processor Intel Corporation

pci@0000:b5:01.4 processor Intel Corporation

pci@0000:b5:01.5 processor Intel Corporation

pci@0000:b5:01.6 processor Intel Corporation

pci@0000:b5:01.7 processor Intel Corporation

pci@0000:b5:02.0 processor Intel Corporation

pci@0000:b5:02.1 processor Intel Corporation

pci@0000:b5:02.2 processor Intel Corporation

pci@0000:b5:02.3 processor Intel Corporation

pci@0000:b5:02.4 processor Intel Corporation

pci@0000:b5:02.5 processor Intel Corporation

pci@0000:b5:02.6 processor Intel Corporation

pci@0000:b5:02.7 processor Intel Corporation

The above example has one physical QAT card with three QAT accelerators:

[root@localhost ~]# lspci | grep Ethernet

86:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

86:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

86:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

88:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

88:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

[root@localhost ~]# lshw -c network -businfo | grep X710

pci@0000:86:00.0 p5p1 network Ethernet Controller X710 for 10GbE SFP+

pci@0000:86:00.1 p5p2 network Ethernet Controller X710 for 10GbE SFP+

pci@0000:88:00.0 p7p1 network Ethernet Controller X710 for 10GbE SFP+

pci@0000:88:00.1 p7p2 network Ethernet Controller X710 for 10GbE SFP+

Resources

Enabling SR-IOV and creating the VF

Setting up SR-IOV on KVM involves creating a PF for each physical network card in the hardware platform. VFs allow FortiGate-VMs to communicate through the physical network card. VFs are actual PCIe hadware resources and only a limited number of VFs are available for each PF. Each QAT addon card creates three PFs with a maximum capacity of 16 VFs each. Ensure that the VM and QAT card are on the same NUMA node.

To configure SR-IOV and create VFs, see SR-IOV support for virtual networking.

After enabling SR-IOV and setting the VF numbers, review the QAT and NIC VFs:

[root@localhost net]# lshw -c processor -businfo

Bus info Device Class Description

========================================================

cpu@0 processor Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz

cpu@1 processor Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz

pci@0000:b1:00.0 processor C62x Chipset QuickAssist Technology

pci@0000:b1:01.0 processor Intel Corporation

pci@0000:b1:01.1 processor Intel Corporation

pci@0000:b1:01.2 processor Intel Corporation

pci@0000:b1:01.3 processor Intel Corporation

pci@0000:b1:01.4 processor Intel Corporation

pci@0000:b1:01.5 processor Intel Corporation

pci@0000:b1:01.6 processor Intel Corporation

pci@0000:b1:01.7 processor Intel Corporation

pci@0000:b1:02.0 processor Intel Corporation

pci@0000:b1:02.1 processor Intel Corporation

pci@0000:b1:02.2 processor Intel Corporation

pci@0000:b1:02.3 processor Intel Corporation

pci@0000:b1:02.4 processor Intel Corporation

pci@0000:b1:02.5 processor Intel Corporation

pci@0000:b1:02.6 processor Intel Corporation

pci@0000:b1:02.7 processor Intel Corporation

pci@0000:b3:00.0 processor C62x Chipset QuickAssist Technology

pci@0000:b3:01.0 processor Intel Corporation

pci@0000:b3:01.1 processor Intel Corporation

pci@0000:b3:01.2 processor Intel Corporation

pci@0000:b3:01.3 processor Intel Corporation

pci@0000:b3:01.4 processor Intel Corporation

pci@0000:b3:01.5 processor Intel Corporation

pci@0000:b3:01.6 processor Intel Corporation

pci@0000:b3:01.7 processor Intel Corporation

pci@0000:b3:02.0 processor Intel Corporation

pci@0000:b3:02.1 processor Intel Corporation

pci@0000:b3:02.2 processor Intel Corporation

pci@0000:b3:02.3 processor Intel Corporation

pci@0000:b3:02.4 processor Intel Corporation

pci@0000:b3:02.5 processor Intel Corporation

pci@0000:b3:02.6 processor Intel Corporation

pci@0000:b3:02.7 processor Intel Corporation

pci@0000:b5:00.0 processor C62x Chipset QuickAssist Technology

pci@0000:b5:01.0 processor Intel Corporation

pci@0000:b5:01.1 processor Intel Corporation

pci@0000:b5:01.2 processor Intel Corporation

pci@0000:b5:01.3 processor Intel Corporation

pci@0000:b5:01.4 processor Intel Corporation

pci@0000:b5:01.5 processor Intel Corporation

pci@0000:b5:01.6 processor Intel Corporation

pci@0000:b5:01.7 processor Intel Corporation

pci@0000:b5:02.0 processor Intel Corporation

pci@0000:b5:02.1 processor Intel Corporation

pci@0000:b5:02.2 processor Intel Corporation

pci@0000:b5:02.3 processor Intel Corporation

pci@0000:b5:02.4 processor Intel Corporation

pci@0000:b5:02.5 processor Intel Corporation

pci@0000:b5:02.6 processor Intel Corporation

pci@0000:b5:02.7 processor Intel Corporation

The above example has one physical QAT card with three QAT accelerators:

[root@localhost ~]# lspci | grep Ethernet

86:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

86:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

86:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

86:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

88:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)

88:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:02.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.2 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.3 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.4 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.5 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

88:0a.7 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02)

[root@localhost ~]# lshw -c network -businfo | grep X710

pci@0000:86:00.0 p5p1 network Ethernet Controller X710 for 10GbE SFP+

pci@0000:86:00.1 p5p2 network Ethernet Controller X710 for 10GbE SFP+

pci@0000:88:00.0 p7p1 network Ethernet Controller X710 for 10GbE SFP+

pci@0000:88:00.1 p7p2 network Ethernet Controller X710 for 10GbE SFP+