DPDK support
You can now enable DPDK on FortiGate-VMs deployed on the Google Cloud Platform. DPDK allows improved network performance.
|
Enabling DPDK in polling mode may result in high CPU usage. For GCP, a VM vCPU is allocated as a dedicated vSPU, resulting in one vCPU's usage to be higher or 100% for IPS in most cases. |
The following example enables DPDK on a FortiGate-VM deployed on Google Cloud, passes UDP and TCP traffic with an antivirus (AV)/IPS/application firewall policy enabled, then checks the engine and vNP statistics.
To enable DPDK on a FortiGate-VM deployed on Google Cloud:
- In the FortiOS CLI, enable DPDK, reboot, then check the DPDK status:
config dpdk global (global) # set status enable (global) # get status : enable interface : multiqueue : disable sleep-on-idle : disable elasticbuffer : disable per-session-accounting: traffic-log-only ipsec-offload : disable hugepage-percentage : 30 mbufpool-percentage : 25 (global) # set interface port1 port2 port3 port4 (global) # set multiqueue enable (global) # set sleep-on-idle enable (global) # set elasticbuffer enable (global) # end status, interface change will trigger system reboot and will take effect after the reboot. Enabling DPDK will adjust Tx/Rx ring size to max allowable value by PMD for the best performance. Do you want to continue? (y/n)y config dpdk global set status enable set interface "port1" "port2" "port3" "port4" set multiqueue enable set sleep-on-idle enable set elasticbuffer enable set per-session-accounting traffic-log-only set ipsec-offload disable set hugepage-percentage 30 set mbufpool-percentage 25 end - Check early initialization logs:
diagnose dpdk log show early-init ----------------------------------------------------------------- DPDK early initialization starts at 2022-03-23 04:58:00(UTC) ----------------------------------------------------------------- Content of DPDK configuration:(Use cmdb configuration) config dpdk global set status enable set interface "port1" "port2" "port3" "port4" set multiqueue enable set sleep-on-idle enable set elasticbuffer enable set per-session-accounting traffic-log-only set ipsec-offload disable set hugepage-percentage 30 set mbufpool-percentage 25 end config dpdk cpus set rx-cpus "all" set vnp-cpus "all" set ips-cpus "all" set tx-cpus "all" end Parse config success! Check CPU definitions 'rx-cpus' Check CPU definitions 'vnp-cpus' Check CPU definitions 'ips-cpus' Check CPU definitions 'tx-cpus' Check CPU definitions 'isolated-cpus' Check CPUs success! Huge page allocation done Ports enabled for DPDK: port1 port2 port3 port4 Port name to device name mapping: port1: eth0 port2: eth1 port3: eth2 port4: eth3 port5: eth4 port6: eth5 port7: eth6 port8: eth7 port9: eth8 port10: eth9 port11: eth10 port12: eth11 port13: eth12 port14: eth13 port15: eth14 port16: eth15 port17: eth16 port18: eth17 port19: eth18 port20: eth19 port21: eth20 port22: eth21 port23: eth22 port24: eth23 Start enabling DPDK kernel driver for port 'port1'... Getting PCI device info for eth0... reading pci dev /sys/class/net/eth0 link path: ../../devices/pci0000:00/0000:00:04.0/virtio1/net/eth0 Device info of eth0: dev_name: eth0 macaddr: 42:01:0a:00:00:0f pci_vendor: 0x1af4 pci_device: 0x1000 pci_id: 0000:00:04.0 pci_domain: 0 pci_bus: 0 pci_devid: 4 pci_function: 0 guid: n/a Unbinding device eth0 from kernel driver... Device eth0 unbind from kernel driver successful Binding device eth0 to DPDK driver... Device eth0 bind to DPDK driver successful Creating DPDK kernel driver for device eth0... Add VNP dev: eth0 PCI: 0000:00:04.0, Succeeded DPDK kernel driver for eth0 successfully created DPDK kernel driver enabled for port 'port1' (device name 'eth0') Start enabling DPDK kernel driver for port 'port2'... Getting PCI device info for eth1... reading pci dev /sys/class/net/eth1 link path: ../../devices/pci0000:00/0000:00:05.0/virtio2/net/eth1 Device info of eth1: dev_name: eth1 macaddr: 42:01:0a:00:01:0f pci_vendor: 0x1af4 pci_device: 0x1000 pci_id: 0000:00:05.0 pci_domain: 0 pci_bus: 0 pci_devid: 5 pci_function: 0 guid: n/a Unbinding device eth1 from kernel driver... Device eth1 unbind from kernel driver successful Binding device eth1 to DPDK driver... Device eth1 bind to DPDK driver successful Creating DPDK kernel driver for device eth1... Add VNP dev: eth1 PCI: 0000:00:05.0, Succeeded DPDK kernel driver for eth1 successfully created DPDK kernel driver enabled for port 'port2' (device name 'eth1') Start enabling DPDK kernel driver for port 'port3'... Getting PCI device info for eth2... reading pci dev /sys/class/net/eth2 link path: ../../devices/pci0000:00/0000:00:06.0/virtio3/net/eth2 Device info of eth2: dev_name: eth2 macaddr: 42:01:0a:00:02:0f pci_vendor: 0x1af4 pci_device: 0x1000 pci_id: 0000:00:06.0 pci_domain: 0 pci_bus: 0 pci_devid: 6 pci_function: 0 guid: n/a Unbinding device eth2 from kernel driver... Device eth2 unbind from kernel driver successful Binding device eth2 to DPDK driver... Device eth2 bind to DPDK driver successful Creating DPDK kernel driver for device eth2... Add VNP dev: eth2 PCI: 0000:00:06.0, Succeeded DPDK kernel driver for eth2 successfully created DPDK kernel driver enabled for port 'port3' (device name 'eth2') Start enabling DPDK kernel driver for port 'port4'... Getting PCI device info for eth3... reading pci dev /sys/class/net/eth3 link path: ../../devices/pci0000:00/0000:00:07.0/virtio4/net/eth3 Device info of eth3: dev_name: eth3 macaddr: 42:01:0a:00:03:0f pci_vendor: 0x1af4 pci_device: 0x1000 pci_id: 0000:00:07.0 pci_domain: 0 pci_bus: 0 pci_devid: 7 pci_function: 0 guid: n/a Unbinding device eth3 from kernel driver... Device eth3 unbind from kernel driver successful Binding device eth3 to DPDK driver... Device eth3 bind to DPDK driver successful Creating DPDK kernel driver for device eth3... Add VNP dev: eth3 PCI: 0000:00:07.0, Succeeded DPDK kernel driver for eth3 successfully created DPDK kernel driver enabled for port 'port4' (device name 'eth3') Bind ports success! mknod for uio0 (254, 0) done. mknod for uio1 (254, 1) done. mknod for uio2 (254, 2) done. mknod for uio3 (254, 3) done. Make UIO nodes success! #---------------EAL INIT----------------- #---------------------------------------- #---------------------------------------- # port oid dev_name pci_id #---------------------------------------- 0 0 eth0 0000:00:04.0 1 1 eth1 0000:00:05.0 2 2 eth2 0000:00:06.0 3 3 eth3 0000:00:07.0 #---------------------------------------- DPDK sanity test passed - Pass UDP and TCP traffic with AV/IPS/application firewall policy enabled, then check engine and vNP statistics:
diagnose dpdk statistics show engine -------------------------------------------------------------------------------- FortiOS DPDK Helper Engine Stats -------------------------------------------------------------------------------- Total Engine 0 Engine 1 Engine 2 Engine 3 CPU ID: 0 1 2 3 ---------- DPDK RX Stage ------------------------------------------------------- dpdkrx_rx_pkts: 2610346 87916 2521121 5 1304 dpdkrx_tx_pkts: 2610346 87916 2521121 5 1304 dpdkrx_drop_pkts: 0 0 0 0 0 dpdkrx_drop_multiseg_pkts: 0 0 0 0 0 dpdkrx_elstcbuf_in_num: 0 0 0 0 0 dpdkrx_elstcbuf_out_num: 0 0 0 0 0 dpdkrx_monitor_rx_cnt: 0 0 0 0 0 ---------- VNP Stage ----------------------------------------------------------- vnp_rx_from_kernel_pkts: 30974 6260 6161 10159 8394 vnp_rx_pkts: 2610346 720505 793687 654737 441417 vnp_tx_pkts: 2608246 723777 788462 653882 442125 vnp_tx_drop_pkts: 0 0 0 0 0 vnp_to_ips_pkts: 2738 885 656 652 545 vnp_to_ips_drop_pkts: 0 0 0 0 0 vnp_to_vnp_pkts: 0 0 0 0 0 vnp_to_vnp_drop_pkts: 0 0 0 0 0 vnp_to_kernel_pkts: 30289 2090 10709 10342 7148 ipsec_dec_pkts: 0 0 0 0 0 ipsec_enc_pkts: 0 0 0 0 0 ipsec_sa_add: 0 0 0 0 0 ipsec_sa_upd: 0 0 0 0 0 ipsec_sa_del: 0 0 0 0 0 ipsec_spi_add: 0 0 0 0 0 ipsec_spi_add_fail: 0 0 0 0 0 ipsec_spi_del: 0 0 0 0 0 ipsec_spi_del_fail: 0 0 0 0 0 ipsec_spi_lookup: 0 0 0 0 0 ipsec_spi_lookup_fail: 0 0 0 0 0 ipsec_spi_reclaim: 0 0 0 0 0 ipsec_ib_sa_hit: 0 0 0 0 0 ipsec_ib_sa_miss: 0 0 0 0 0 ipsec_ib_headroom_err: 0 0 0 0 0 ipsec_ib_cryptodev_err: 0 0 0 0 0 ipsec_ib_post_proc_err: 0 0 0 0 0 ipsec_ib_uesp_dport_err: 0 0 0 0 0 ipsec_ib_uesp_not_enabled: 0 0 0 0 0 ipsec_ob_sa_hit: 0 0 0 0 0 ipsec_ob_sa_miss: 0 0 0 0 0 ipsec_ob_headroom_err: 0 0 0 0 0 ipsec_ob_cryptodev_err: 0 0 0 0 0 ipsec_ob_post_proc_err: 0 0 0 0 0 ---------- IPS Stage ----------------------------------------------------------- ips_rx_pkts: 2738 657 698 705 678 ips_tx_pkts: 2738 657 698 705 678 ips_drop_pkts: 0 0 0 0 0 ips_vdct_pkts: 0 0 0 0 0 ips_inv_pkts: 0 0 0 0 0 from_ips_rx_pkts: 0 0 0 0 0 from_ips_tx_pkts: 0 0 0 0 0 from_ips_drop_pkts: 0 0 0 0 0 from_ips_fallback_pkts: 0 0 0 0 0 ---------- DPDK TX Stage ------------------------------------------------------- dpdktx_rx_pkts: 2610984 2522231 86925 893 935 dpdktx_tx_pkts: 2610984 2522231 86925 893 935 dpdktx_drop_pkts: 0 0 0 0 0 dpdktx_drop_oversized_pkt: 0 0 0 0 0 diagnose dpdk statistics show vnp -------------------------------------------------------------------------------- FortiOS DPDK Helper VNP Stats -------------------------------------------------------------------------------- Total Engine 0 Engine 1 Engine 2 Engine 3 CPU ID: 0 1 2 3 ---------- VNP Internal -------------------------------------------------------- ctr_sse: 224038 68362 49639 50015 56022 ctr_sse_cmd: 168 62 34 39 33 ctr_sse_delmiss: 0 0 0 0 0 ctr_sse_msg: 113 48 11 17 37 ctr_sse_pruned: 0 0 0 0 0 vnp_st_rx_from_dpdkrx: 2610531 720527 793688 654804 441512 vnp_st_sse_proc: 2582749 718835 783299 644776 435839 vnp_st_tx_to_kernel: 30474 2112 10710 10409 7243 vnp_st_ipsec_ib: 0 0 0 0 0 vnp_st_ipsec_ob: 0 0 0 0 0 vnp_st_fpath_proc: 2580170 718463 782989 644412 434306 vnp_st_tx_to_dpdktx: 2608486 723802 788463 653996 442225 vnp_st_tx_to_ips: 2738 885 656 652 545 vnp_st_rx_from_kernel: 31222 6286 6164 10275 8497 vnp_st_sse_cmd: 168 62 34 39 33 vnp_st_final: 33380 3059 11400 11100 7821 ctr_sse_entries: 8 2 2 3 1 err_sse_batch_size: 0 0 0 0 0 err_sse_unknown_cmd: 0 0 0 0 0 err_sse_full: 0 0 0 0 0 err_sse_tbl_alloc_fail: 0 0 0 0 0 err_sse_inv_oid: 0 0 0 0 0 err_fp_no_act: 0 0 0 0 0 err_fp_no_port: 0 0 0 0 0 drop_inv_l3: 0 0 0 0 0 drop_inv_l4: 0 0 0 0 0 drop_fp_act: 0 0 0 0 0 drop_inv_port: 0 0 0 0 0 drop_inv_ip_cksum: 0 0 0 0 0 drop_oversized_pkt: 0 0 0 0 0 drop_unsupported: 0 0 0 0 0 drop_looping_pkt: 0 0 0 0 0 drop_ipsec_ob_fail: 0 0 0 0 0 --------------------------------------------------------------------------------