SD-WAN hub and spoke speed test improvements 7.4.2
This information is also available in the FortiOS 7.4 Administration Guide: |
SD-WAN hub and spoke speed tests include the following improvements:
-
Speed test servers can be deployed on a hub or a spoke. When deployed on a hub, speed tests can be initiated from spokes, even when a spoke is behind a NAT device.
-
Tests can be in upload or download direction.
-
Both TCP and UDP protocols are supported.
-
An egress-shaping profile can be applied to local, remote, or both local and remote IPsec tunnels or no IPsec tunnels.
-
Custom speed-test listening ports can be configured.
The test measures the speeds of the link to each spoke so that QoS can be applied on the hub to the dynamic IPsec overlay tunnels to each spoke. When the speed test is initiated from the spoke, the results are cached on the spoke, but sent to the hub to be applied to the egress traffic shaping profile assigned to the IPsec overlay tunnel interface and the respective tunnel. For more information about SD-WAN hub and spoke speed tests, see Running speed tests from the hub to the spokes in dial-up IPsec tunnels.
When a speed-test server is enabled, two speed test daemons are started and listen on different ports for different purposes:
-
The controller speed test daemon listens on the IPsec overlay interfaces to assign an access token to each incoming speed test for authentication.
-
The speed test daemon listens on the IPsec underlay interfaces to handle the speed tests.
Each incoming speed test request must present the obtained access token to prevent random, unauthorized requests. Otherwise, the connection is closed immediately. As such, speed test access must be enabled on both the underlay and the IPsec overlay tunnel interfaces on the hub.
config system interface edit <interface> set allowaccess speed-test [other access] ... next end
If the IPsec tunnel has a configured |
New commands are available to configure custom speed-test listening ports for the speed test server:
config system global set speedtestd-server-port <integer> set speedtestd-ctrl-port <integer> end
set speedtestd-server-port <integer> |
Specify a custom port number (1024 - 65535, default = 5201) for the speed test daemon. The port is used to perform the speed test. |
set speedtestd-ctrl-port <integer> |
Specify a custom port number (1024 - 65535. default = 5200) for the controller speed test daemon. The port is used to assign access tokens for authentication prior to performing the speed test. |
The speed test client can be a hub or a spoke and must have system speed-test-schedule
configured and the dynamic-server
setting enabled.
On the speed test client, specify whether and how to apply the test results in a shaping profile. The shaping profile must be configured in the phase1 interface before it can be used with a speed test.
config system speed-test-schedule edit <interface> set server-port <integer> set ctrl-port <integer> set update-shaper {disable | local | remote | both} next end
set server-port <integer> |
Specify the port number for the speed-test server used for speed tests (1 - 65535, default = 5201). |
set ctrl-port <integer> |
Specify the port number for the controller on the speed-test server used for authentication (1 - 65535, default = 5200). |
set update-shaper {disable | local | remote | both} |
Set the egress shaper to use the speed test results:
|
Example
In this hub and spoke example, the hub is configured as an IPsec VPN dial-up server with two IPsec tunnels, and each tunnel is connected to a spoke. The VPN interfaces and IP addresses are:
FortiGate |
Interface |
IP Address |
---|---|---|
FGT_A (Hub) |
hub-phase1 |
10.10.15.253 |
FGT_B (Spoke) |
spoke11-p1 |
10.10.15.2 |
FGT_D (Spoke) |
spoke21-p1 |
10.10.15.1 |
The hub (FGT_A) is configured as a speed-test server to listen on custom ports (6000 and 7000), and the spokes (FGT_B and FGT_D) are configured as speed-test clients. This setup allows speed tests to successfully perform when spokes are behind NAT devices. The results of the speed test will be applied to the hub-phase1 overlay tunnel(s) as specified by the speed-test clients.
The spokes are configured to initiate speed tests on a schedule on UDP. After the speed test completes, the results are sent to the hub, and the hub applies the results on its IPsec tunnels as egress traffic shaping. The results are also cached and can be used if an IPsec tunnel is disconnected and reconnected again.
This example focuses on the key settings required to enable a hub as the speed-test server and the spokes as speed-test clients that initiate the speed tests. For a complete example about running speed tests from the hub, see Running speed tests from the hub to the spokes in dial-up IPsec tunnels. |
To configure the hub FortiGate (FGT_A):
-
Configure a shaping profile:
In this example, the shaping profile is named profile_1.
config firewall shaping-profile edit "profile_1" set default-class-id 2 config shaping-entries edit 1 set class-id 2 set priority low set guaranteed-bandwidth-percentage 10 set maximum-bandwidth-percentage 10 next edit 2 set class-id 3 set priority medium set guaranteed-bandwidth-percentage 30 set maximum-bandwidth-percentage 40 next edit 3 set class-id 4 set guaranteed-bandwidth-percentage 20 set maximum-bandwidth-percentage 60 next end end end
Three classes are used in the profile for low, medium, and high priority traffic. Each class is assigned a guaranteed and maximum bandwidth as a percentage of the measured bandwidth from the speed test.
-
Configure a shaping policy to assign certain traffic as a class ID:
In this example, all traffic destined to the dialup tunnels are assigned class 3.
config firewall shaping-policy edit 2 set service "ALL" set schedule "always" set dstintf "hub-phase1" "hub2-phase1" set class-id 3 set srcaddr "all" set dstaddr "all" next end
-
Enable a speed test server with custom speed-test listening ports:
A speed test server is enabled on the hub. Port 7000 will run speed tests, and port 6000 will be the controller used to issue access tokens for speed test authentication.
config system global ... set speedtest-server enable set speedtestd-ctrl-port 6000 set speedtestd-server-port 7000 end
-
Allow the speed test on the underlay:
config system interface edit "port1" set ip 172.16.200.1 255.255.255.0 set allowaccess ping https ssh snmp http telnet fgfm radius-acct probe-response fabric speed-test ... next end
-
Allow the speed test on the overlay and use the shaping profile in the interface:
In this example, speed tests are allowed on the overlay, and the shaping profile (profile_1) is used on the hub phase1 interface (port1).
config system interface edit "hub-phase1" set ip 10.10.15.253 255.255.255.255 set allowaccess ping speed-test set egress-shaping-profile "profile_1" ... set interface "port1" next end
To configure the first spoke FortiGate (FGT_B):
-
Configure
system speed-test-schedule
:The protocol mode is set to UDP. The custom controller port used for authentication is set to 6000, and the custom port used to run the speed tests is set to 7000. The shaping profile is set to remote.
config system speed-test-schedule edit "spoke11-p1" set mode UDP set schedules "1" set dynamic-server enable set ctrl-port 6000 set server-port 7000 set update-shaper remote next end
-
Configure a recurring schedule for the speed tests:
Schedule 1 is set to start at 08:37 every day of the week.
config firewall schedule recurring edit "1" set start 08:37 set day sunday monday tuesday wednesday thursday friday saturday next end
To configure the second spoke FortiGate (FGT_D):
-
Configure a speed test schedule:
The protocol mode is set to UDP. The custom controller port used for authentication is set to 6000, and the custom port used to run the speed tests is set to 7000. The shaping profile is set to remote.
config system speed-test-schedule edit "spoke21-p1" set mode UDP set schedules "1" set dynamic-server enable set ctrl-port 6000 set server-port 7000 set update-shaper remote next end
-
Configure a recurring schedule for the speed tests:
Schedule 1 is set to start at 08:37 every day of the week.
config firewall schedule recurring edit "1" set start 08:37 set day sunday monday tuesday wednesday thursday friday saturday next end
To view the speed test results:
-
After the speed test schedule runs, view the result on spoke FGT_B:
On spoke FGT_B, authentication succeeds through port 6000, and the test runs on port 7000. UDP mode is used, and the test is successful.
# diagnose debug application speedtest -1 ...... fcron_speedtest_ipsec_request_init()-464: root: spoke11-p1(spoke11-p1) id=003900d5 fd=24, init request=0.0.0.0:0 -> 10.10.15.253:6000, test=172.16.200.2:0 -> 172.16.200.1:7000: succeed. ...... [speedtest(2181)] start uploading test. [speedtest(2181)] Connecting to host 172.16.200.1, port 7000 [speedtest(2181)] [ 26] local 172.16.200.2 port 17553 connected to 172.16.200.1 port 7000 [speedtest(2181)] [ ID] Interval Transfer Bitrate Total Datagrams [speedtest(2181)] [ 26] 0.00-1.00 sec 150 MBytes 1.26 Gbits/sec 107570 [speedtest(2181)] [ 26] 1.00-2.00 sec 149 MBytes 1.25 Gbits/sec 107120 [speedtest(2181)] [ 26] 2.00-3.00 sec 149 MBytes 1.25 Gbits/sec 107030 [speedtest(2181)] [ 26] 3.00-4.00 sec 149 MBytes 1.25 Gbits/sec 107210 [speedtest(2181)] [ 26] 4.00-5.00 sec 149 MBytes 1.25 Gbits/sec 107260 [speedtest(2181)] [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [speedtest(2181)] [ 26] 0.00-5.00 sec 747 MBytes 1.25 Gbits/sec 0.000 ms 0/536190 (0%) sender [speedtest(2181)] [ 26] 0.00-5.00 sec 271 MBytes 454 Mbits/sec 0.000 ms 341627/535995 (64%) receiver [speedtest(2181)] client(sender): bytes_recv=283777280, bytes_sent=782837400, sender_time=5.000, recver_time=5.000 [speedtest(2181)] client(sender): up_speed: 454 Mbits/sec [speedtest(2181)] [speedtest(2181)] speed test Done. [speedtest(2181)] start downloading test. [speedtest(2181)] Connecting to host 172.16.200.1, port 7000 [speedtest(2181)] Reverse mode, remote host 172.16.200.1 is sending [speedtest(2181)] [ 26] local 172.16.200.2 port 7998 connected to 172.16.200.1 port 7000 [speedtest(2181)] [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [speedtest(2181)] [ 26] 0.00-1.00 sec 54.6 MBytes 458 Mbits/sec 0.007 ms 70745/109978 (64%) [speedtest(2181)] [ 26] 1.00-2.00 sec 54.8 MBytes 460 Mbits/sec 0.008 ms 67547/106917 (63%) [speedtest(2181)] [ 26] 2.00-3.00 sec 54.9 MBytes 460 Mbits/sec 0.010 ms 67543/106940 (63%) [speedtest(2181)] [ 26] 3.00-4.00 sec 54.8 MBytes 460 Mbits/sec 0.006 ms 67636/107024 (63%) [speedtest(2181)] [ 26] 4.00-5.00 sec 54.9 MBytes 460 Mbits/sec 0.004 ms 67421/106842 (63%) [speedtest(2181)] [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [speedtest(2181)] [ 26] 0.00-5.00 sec 750 MBytes 1.26 Gbits/sec 0.000 ms 0/538540 (0%) sender [speedtest(2181)] [ 26] 0.00-5.00 sec 274 MBytes 460 Mbits/sec 0.004 ms 340892/537701 (63%) receiver [speedtest(2181)] client(recver): bytes_recv=287341140, bytes_sent=786268400, sender_time=5.000, recver_time=5.001 [speedtest(2181)] client(recver): down_speed: 460 Mbits/sec [speedtest(2181)] [speedtest(2181)] speed test Done. fcron_speedtest_notify_func()-1275: Speed test pid=2181 done fcron_speedtest_on_test_finish()-1211: Test 3900d5 for 'spoke11-p1' succeed with up=454043, down=459694 fcron_speedtest_save_results()-1144: Write logs to disk: succ=1, fail=0 fcron_speedtest_sync_results()-1172: Sync cached results to secondary devices.
-
After the speed test schedule runs, view the result on the spoke FGT_D:
On spoke FGT_D, authentication succeeds through port 6000, and the test runs on port 7000. UDP mode is used, and the test is successful.
# diagnose debug application speedtest -1 ...... fcron_speedtest_ipsec_request_init()-464: root: spoke21-p1(spoke21-p1) id=00380011 fd=25, init request=0.0.0.0:0 -> 10.10.15.253:6000, test=172.16.200.4:0 -> 172.16.200.1:7000: succeed. ...... [speedtest(4309)] start uploading test. [speedtest(4309)] Connecting to host 172.16.200.1, port 7000 [speedtest(4309)] [ 27] local 172.16.200.4 port 15349 connected to 172.16.200.1 port 7000 [speedtest(4309)] [ ID] Interval Transfer Bitrate Total Datagrams [speedtest(4309)] [ 27] 0.00-1.00 sec 148 MBytes 1.24 Gbits/sec 105940 [speedtest(4309)] [ 27] 1.00-2.00 sec 148 MBytes 1.24 Gbits/sec 105990 [speedtest(4309)] [ 27] 2.00-3.00 sec 147 MBytes 1.24 Gbits/sec 105860 [speedtest(4309)] [ 27] 3.00-4.00 sec 148 MBytes 1.24 Gbits/sec 105960 [speedtest(4309)] [ 27] 4.00-5.00 sec 148 MBytes 1.24 Gbits/sec 106090 [speedtest(4309)] [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [speedtest(4309)] [ 27] 0.00-5.00 sec 738 MBytes 1.24 Gbits/sec 0.000 ms 0/529840 (0%) sender [speedtest(4309)] [ 27] 0.00-5.00 sec 271 MBytes 454 Mbits/sec 0.000 ms 335130/529650 (63%) receiver [speedtest(4309)] client(sender): bytes_recv=283999200, bytes_sent=773566400, sender_time=5.000, recver_time=5.000 [speedtest(4309)] client(sender): up_speed: 454 Mbits/sec [speedtest(4309)] [speedtest(4309)] speed test Done. [speedtest(4309)] start downloading test. [speedtest(4309)] Connecting to host 172.16.200.1, port 7000 [speedtest(4309)] Reverse mode, remote host 172.16.200.1 is sending [speedtest(4309)] [ 27] local 172.16.200.4 port 19586 connected to 172.16.200.1 port 7000 [speedtest(4309)] [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [speedtest(4309)] [ 27] 0.00-1.00 sec 56.1 MBytes 471 Mbits/sec 0.005 ms 70258/110574 (64%) [speedtest(4309)] [ 27] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0.006 ms 66496/106740 (62%) [speedtest(4309)] [ 27] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0.005 ms 66481/106736 (62%) [speedtest(4309)] [ 27] 3.00-4.00 sec 56.1 MBytes 471 Mbits/sec 0.007 ms 66403/106690 (62%) [speedtest(4309)] [ 27] 4.00-5.00 sec 56.3 MBytes 473 Mbits/sec 0.008 ms 65991/106454 (62%) [speedtest(4309)] [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [speedtest(4309)] [ 27] 0.00-5.00 sec 749 MBytes 1.26 Gbits/sec 0.000 ms 0/538110 (0%) sender [speedtest(4309)] [ 27] 0.00-5.00 sec 281 MBytes 471 Mbits/sec 0.008 ms 335629/537194 (62%) receiver [speedtest(4309)] client(recver): bytes_recv=294284900, bytes_sent=785640600, sender_time=5.000, recver_time=5.001 [speedtest(4309)] client(recver): down_speed: 471 Mbits/sec [speedtest(4309)] [speedtest(4309)] speed test Done. fcron_speedtest_notify_func()-1275: Speed test pid=4309 done fcron_speedtest_on_test_finish()-1211: Test 380011 for 'spoke21-p1' succeed with up=454398, down=470794 fcron_speedtest_save_results()-1144: Write logs to disk: succ=1, fail=0 fcron_speedtest_sync_results()-1172: Sync cached results to secondary devices.
-
After the speed test schedule runs, view the result on the hub (FGT_A):
The server side uses
speedtestd
, while the client side usesspeedtest
.The speed test results are applied on hub-phase1_0 and hub_phase1_1 as egress traffic shaping.
# diagnose debug application speedtestd -1 ...... [speedtest(2771)] [ 7] local 172.16.200.1 port 7000 connected to 172.16.200.2 port 17553 ...... [speedtest(2771)] [ 7] local 172.16.200.1 port 7000 connected to 172.16.200.2 port 7998 ...... [sptestd::ctrl(0377):root] set shaper: if=hub-phase1, tun=hub-phase1_0, sp=profile_1, bw=459745 ...... [speedtest(2771)] [ 7] local 172.16.200.1 port 7000 connected to 172.16.200.4 port 15349 ...... [speedtest(2771)] [ 7] local 172.16.200.1 port 7000 connected to 172.16.200.4 port 19586 ...... [sptestd::ctrl(0377):root] set shaper: if=hub-phase1, tun=hub-phase1_1, sp=profile_1, bw=470855 ......
-
Verify the result is cached on the spokes.
-
On FGT_B, the speed test results are cached:
# diagnose test application forticron 10 Speed test results: 1: vdom=root, phase1intf=spoke11-p1, peer-id='172.16.200.1', up=454043, dw=459694, time=12/13 12:32:19
-
On FGT_D, the speed test results are cached:
# diagnose test application forticron 10 Speed test results: 1: vdom=root, phase1intf=spoke21-p1, peer-id='172.16.200.1', up=454398, dw=470794, time=12/12 16:33:18
-
-
On the hub (FGT_A), verify the speed test results are applied to the hub's IPsec tunnels as egress traffic shaping:
On hub-phase1_0 and hub-phase1_1, the correct traffic control is displayed.
# diagnose vpn tunnel list list all ipsec tunnel in vd 0 ...... ------------------------------------------------------ name=hub-phase1_0 ver=2 serial=16 172.16.200.1:0->172.16.200.2:0 tun_id=10.10.15.1 tun_id6=2000:10:10:15::1 dst_mtu=1500 dpd-link=on weight=1 bound_if=11 lgwy=static/1 tun=intf mode=dial_inst/3 encap=none/74408 options[122a8]=npu rgwy-chg frag-rfc run_state=0 role=primary accept_traffic=1 overlay_id=10 parent=hub-phase1 index=0 ...... egress traffic control: bandwidth=459745(kbps) lock_hit=0 default_class=2 n_active_class=3 class-id=2 allocated-bandwidth=45974(kbps) guaranteed-bandwidth=45974(kbps) max-bandwidth=45974(kbps) current-bandwidth=0(kbps) priority=low forwarded_bytes=86K dropped_packets=0 dropped_bytes=0 class-id=3 allocated-bandwidth=137923(kbps) guaranteed-bandwidth=137923(kbps) max-bandwidth=183897(kbps) current-bandwidth=0(kbps) priority=medium forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 class-id=4 allocated-bandwidth=275846(kbps) guaranteed-bandwidth=91948(kbps) max-bandwidth=275846(kbps) current-bandwidth=0(kbps) priority=high forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 ------------------------------------------------------ name=hub-phase1_1 ver=2 serial=17 172.16.200.1:0->172.16.200.4:0 tun_id=10.10.15.2 tun_id6=2000:10:10:15::2 dst_mtu=1500 dpd-link=on weight=1 bound_if=11 lgwy=static/1 tun=intf mode=dial_inst/3 encap=none/74408 options[122a8]=npu rgwy-chg frag-rfc run_state=0 role=primary accept_traffic=1 overlay_id=10 parent=hub-phase1 index=1 ...... egress traffic control: bandwidth=470855(kbps) lock_hit=0 default_class=2 n_active_class=3 class-id=2 allocated-bandwidth=47085(kbps) guaranteed-bandwidth=47085(kbps) max-bandwidth=47085(kbps) current-bandwidth=0(kbps) priority=low forwarded_bytes=81K dropped_packets=0 dropped_bytes=0 class-id=3 allocated-bandwidth=141256(kbps) guaranteed-bandwidth=141256(kbps) max-bandwidth=188341(kbps) current-bandwidth=0(kbps) priority=medium forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 class-id=4 allocated-bandwidth=282512(kbps) guaranteed-bandwidth=94170(kbps) max-bandwidth=282512(kbps) current-bandwidth=0(kbps) priority=high forwarded_bytes=0 dropped_packets=0 dropped_bytes=0