SD-WAN hub and spoke speed test improvements 7.4.2

SD-WAN hub and spoke speed tests include the following improvements:

• Speed test servers can be deployed on a hub or a spoke. When deployed on a hub, speed tests can be initiated from spokes, even when a spoke is behind a NAT device.

• Tests can be in upload or download direction.

• Both TCP and UDP protocols are supported.

• An egress-shaping profile can be applied to local, remote, or both local and remote IPsec tunnels or no IPsec tunnels.

• Custom speed-test listening ports can be configured.

The test measures the speeds of the link to each spoke so that QoS can be applied on the hub to the dynamic IPsec overlay tunnels to each spoke. When the speed test is initiated from the spoke, the results are cached on the spoke, but sent to the hub to be applied to the egress traffic shaping profile assigned to the IPsec overlay tunnel interface and the respective tunnel. For more information about SD-WAN hub and spoke speed tests, see Running speed tests from the hub to the spokes in dial-up IPsec tunnels.

When a speed-test server is enabled, two speed test daemons are started and listen on different ports for different purposes:

• The controller speed test daemon listens on the IPsec overlay interfaces to assign an access token to each incoming speed test for authentication.

• The speed test daemon listens on the IPsec underlay interfaces to handle the speed tests.

Each incoming speed test request must present the obtained access token to prevent random, unauthorized requests. Otherwise, the connection is closed immediately. As such, speed test access must be enabled on both the underlay and the IPsec overlay tunnel interfaces on the hub.

config system interface
    edit <interface>
        set allowaccess speed-test [other access] ...
    next
end

If the IPsec tunnel has a configured exchange-ip, speed test access must also be configured on the associated interface, such as the loopback interface.

New commands are available to configure custom speed-test listening ports for the speed test server:

config system global
    set speedtestd-server-port <integer>
    set speedtestd-ctrl-port <integer>
end

set speedtestd-server-port <integer>	Specify a custom port number (1024 - 65535, default = 5201) for the speed test daemon. The port is used to perform the speed test.
set speedtestd-ctrl-port <integer>	Specify a custom port number (1024 - 65535. default = 5200) for the controller speed test daemon. The port is used to assign access tokens for authentication prior to performing the speed test.

The speed test client can be a hub or a spoke and must have system speed-test-schedule configured and the dynamic-server setting enabled.

On the speed test client, specify whether and how to apply the test results in a shaping profile. The shaping profile must be configured in the phase1 interface before it can be used with a speed test.

config system speed-test-schedule
    edit <interface>
        set server-port <integer>
        set ctrl-port <integer>
        set update-shaper {disable | local | remote | both}
    next
end

set server-port <integer>	Specify the port number for the speed-test server used for speed tests (1 - 65535, default = 5201).
set ctrl-port <integer>	Specify the port number for the controller on the speed-test server used for authentication (1 - 65535, default = 5200).
set update-shaper {disable | local | remote | both}	Set the egress shaper to use the speed test results: disable: Disable updating the egress shaper (default). local: Update the speed-test client egress shaper. remote: Update the speed-test server egress shaper. both: Update both the local and remote egress shapers.

Example

In this hub and spoke example, the hub is configured as an IPsec VPN dial-up server with two IPsec tunnels, and each tunnel is connected to a spoke. The VPN interfaces and IP addresses are:

FortiGate	Interface	IP Address
FGT_A (Hub)	hub-phase1	10.10.15.253
FGT_B (Spoke)	spoke11-p1	10.10.15.2
FGT_D (Spoke)	spoke21-p1	10.10.15.1

The hub (FGT_A) is configured as a speed-test server to listen on custom ports (6000 and 7000), and the spokes (FGT_B and FGT_D) are configured as speed-test clients. This setup allows speed tests to successfully perform when spokes are behind NAT devices. The results of the speed test will be applied to the hub-phase1 overlay tunnel(s) as specified by the speed-test clients.

The spokes are configured to initiate speed tests on a schedule on UDP. After the speed test completes, the results are sent to the hub, and the hub applies the results on its IPsec tunnels as egress traffic shaping. The results are also cached and can be used if an IPsec tunnel is disconnected and reconnected again.

This example focuses on the key settings required to enable a hub as the speed-test server and the spokes as speed-test clients that initiate the speed tests. For a complete example about running speed tests from the hub, see Running speed tests from the hub to the spokes in dial-up IPsec tunnels.

To configure the hub FortiGate (FGT_A):

1. Configure a shaping profile:

   In this example, the shaping profile is named profile_1.

   config firewall shaping-profile
       edit "profile_1"
           set default-class-id 2
           config shaping-entries
               edit 1
                   set class-id 2
                   set priority low
                   set guaranteed-bandwidth-percentage 10
                   set maximum-bandwidth-percentage 10
               next
               edit 2
                   set class-id 3
                   set priority medium
                   set guaranteed-bandwidth-percentage 30
                   set maximum-bandwidth-percentage 40
               next
               edit 3
                   set class-id 4
                   set guaranteed-bandwidth-percentage 20
                   set maximum-bandwidth-percentage 60
               next
           end
       end
   end

   Three classes are used in the profile for low, medium, and high priority traffic. Each class is assigned a guaranteed and maximum bandwidth as a percentage of the measured bandwidth from the speed test.

2. Configure a shaping policy to assign certain traffic as a class ID:

   In this example, all traffic destined to the dialup tunnels are assigned class 3.

   config firewall shaping-policy
       edit 2
           set service "ALL"
           set schedule "always"
           set dstintf "hub-phase1" "hub2-phase1"
           set class-id 3
           set srcaddr "all"
           set dstaddr "all"
       next
   end

3. Enable a speed test server with custom speed-test listening ports:

   A speed test server is enabled on the hub. Port 7000 will run speed tests, and port 6000 will be the controller used to issue access tokens for speed test authentication.

   config system global
       ...
       set speedtest-server enable 
       set speedtestd-ctrl-port 6000 
       set speedtestd-server-port 7000
   end

4. Allow the speed test on the underlay:

   config system interface
       edit "port1"
           set ip 172.16.200.1 255.255.255.0
           set allowaccess ping https ssh snmp http telnet fgfm radius-acct probe-response fabric speed-test
           ...
       next
   end

5. Allow the speed test on the overlay and use the shaping profile in the interface:

   In this example, speed tests are allowed on the overlay, and the shaping profile (profile_1) is used on the hub phase1 interface (port1).

   config system interface                
       edit "hub-phase1" 
           set ip 10.10.15.253 255.255.255.255
           set allowaccess ping speed-test      
           set egress-shaping-profile "profile_1" 
           ...
           set interface "port1"
       next
   end

To configure the first spoke FortiGate (FGT_B):

1. Configure system speed-test-schedule:

   The protocol mode is set to UDP. The custom controller port used for authentication is set to 6000, and the custom port used to run the speed tests is set to 7000. The shaping profile is set to remote.

   config system speed-test-schedule
       edit "spoke11-p1"
           set mode UDP
           set schedules "1"
           set dynamic-server enable
           set ctrl-port 6000
           set server-port 7000
           set update-shaper remote
       next
   end

2. Configure a recurring schedule for the speed tests:

   Schedule 1 is set to start at 08:37 every day of the week.

   config firewall schedule recurring
       edit "1"
           set start 08:37
           set day sunday monday tuesday wednesday thursday friday saturday
       next
   end

To configure the second spoke FortiGate (FGT_D):

1. Configure a speed test schedule:

   The protocol mode is set to UDP. The custom controller port used for authentication is set to 6000, and the custom port used to run the speed tests is set to 7000. The shaping profile is set to remote.

   config system speed-test-schedule
       edit "spoke21-p1"
           set mode UDP
           set schedules "1"
           set dynamic-server enable
           set ctrl-port 6000
           set server-port 7000
           set update-shaper remote
       next
   end

2. Configure a recurring schedule for the speed tests:

   Schedule 1 is set to start at 08:37 every day of the week.

   config firewall schedule recurring
       edit "1"
           set start 08:37
           set day sunday monday tuesday wednesday thursday friday saturday
       next
   end

To view the speed test results:

1. After the speed test schedule runs, view the result on spoke FGT_B:

   On spoke FGT_B, authentication succeeds through port 6000, and the test runs on port 7000. UDP mode is used, and the test is successful.

   # diagnose debug application speedtest -1
   
   ......
   fcron_speedtest_ipsec_request_init()-464: root: spoke11-p1(spoke11-p1) id=003900d5 fd=24, init request=0.0.0.0:0 -> 10.10.15.253:6000, test=172.16.200.2:0 -> 172.16.200.1:7000: succeed.
   ......
   [speedtest(2181)] start uploading test.
   [speedtest(2181)] Connecting to host 172.16.200.1, port 7000
   [speedtest(2181)] [ 26] local 172.16.200.2 port 17553 connected to 172.16.200.1 port 7000
   [speedtest(2181)] [ ID] Interval           Transfer     Bitrate         Total Datagrams
   [speedtest(2181)] [ 26]   0.00-1.00   sec   150 MBytes  1.26 Gbits/sec  107570
   [speedtest(2181)] [ 26]   1.00-2.00   sec   149 MBytes  1.25 Gbits/sec  107120
   [speedtest(2181)] [ 26]   2.00-3.00   sec   149 MBytes  1.25 Gbits/sec  107030
   [speedtest(2181)] [ 26]   3.00-4.00   sec   149 MBytes  1.25 Gbits/sec  107210
   [speedtest(2181)] [ 26]   4.00-5.00   sec   149 MBytes  1.25 Gbits/sec  107260
   [speedtest(2181)] [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
   [speedtest(2181)] [ 26]   0.00-5.00   sec   747 MBytes  1.25 Gbits/sec  0.000 ms  0/536190 (0%)  sender
   [speedtest(2181)] [ 26]   0.00-5.00   sec   271 MBytes   454 Mbits/sec  0.000 ms  341627/535995 (64%)  receiver
   [speedtest(2181)] client(sender): bytes_recv=283777280, bytes_sent=782837400, sender_time=5.000, recver_time=5.000
   [speedtest(2181)] client(sender): up_speed:  454 Mbits/sec
   [speedtest(2181)]
   [speedtest(2181)] speed test Done.       
   [speedtest(2181)] start downloading test. 
   [speedtest(2181)] Connecting to host 172.16.200.1, port 7000
   [speedtest(2181)] Reverse mode, remote host 172.16.200.1 is sending
   [speedtest(2181)] [ 26] local 172.16.200.2 port 7998 connected to 172.16.200.1 port 7000
   [speedtest(2181)] [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
   [speedtest(2181)] [ 26]   0.00-1.00   sec  54.6 MBytes   458 Mbits/sec  0.007 ms  70745/109978 (64%)
   [speedtest(2181)] [ 26]   1.00-2.00   sec  54.8 MBytes   460 Mbits/sec  0.008 ms  67547/106917 (63%)
   [speedtest(2181)] [ 26]   2.00-3.00   sec  54.9 MBytes   460 Mbits/sec  0.010 ms  67543/106940 (63%)
   [speedtest(2181)] [ 26]   3.00-4.00   sec  54.8 MBytes   460 Mbits/sec  0.006 ms  67636/107024 (63%)
   [speedtest(2181)] [ 26]   4.00-5.00   sec  54.9 MBytes   460 Mbits/sec  0.004 ms  67421/106842 (63%)
   [speedtest(2181)] [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
   [speedtest(2181)] [ 26]   0.00-5.00   sec   750 MBytes  1.26 Gbits/sec  0.000 ms  0/538540 (0%)  sender
   [speedtest(2181)] [ 26]   0.00-5.00   sec   274 MBytes   460 Mbits/sec  0.004 ms  340892/537701 (63%)  receiver
   [speedtest(2181)] client(recver): bytes_recv=287341140, bytes_sent=786268400, sender_time=5.000, recver_time=5.001
   [speedtest(2181)] client(recver): down_speed:  460 Mbits/sec
   [speedtest(2181)]
   [speedtest(2181)] speed test Done.
   fcron_speedtest_notify_func()-1275: Speed test pid=2181 done
   
   fcron_speedtest_on_test_finish()-1211: Test 3900d5 for 'spoke11-p1' succeed with up=454043, down=459694
   fcron_speedtest_save_results()-1144: Write logs to disk: succ=1, fail=0
   fcron_speedtest_sync_results()-1172: Sync cached results to secondary devices.

2. After the speed test schedule runs, view the result on the spoke FGT_D:

   On spoke FGT_D, authentication succeeds through port 6000, and the test runs on port 7000. UDP mode is used, and the test is successful.

   # diagnose debug application speedtest -1
   
   ......
   fcron_speedtest_ipsec_request_init()-464: root: spoke21-p1(spoke21-p1) id=00380011 fd=25, init request=0.0.0.0:0 -> 10.10.15.253:6000, test=172.16.200.4:0 -> 172.16.200.1:7000: succeed.
   ...... 
   [speedtest(4309)] start uploading test.
   [speedtest(4309)] Connecting to host 172.16.200.1, port 7000
   [speedtest(4309)] [ 27] local 172.16.200.4 port 15349 connected to 172.16.200.1 port 7000
   [speedtest(4309)] [ ID] Interval           Transfer     Bitrate         Total Datagrams
   [speedtest(4309)] [ 27]   0.00-1.00   sec   148 MBytes  1.24 Gbits/sec  105940
   [speedtest(4309)] [ 27]   1.00-2.00   sec   148 MBytes  1.24 Gbits/sec  105990
   [speedtest(4309)] [ 27]   2.00-3.00   sec   147 MBytes  1.24 Gbits/sec  105860
   [speedtest(4309)] [ 27]   3.00-4.00   sec   148 MBytes  1.24 Gbits/sec  105960
   [speedtest(4309)] [ 27]   4.00-5.00   sec   148 MBytes  1.24 Gbits/sec  106090
   [speedtest(4309)] [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
   [speedtest(4309)] [ 27]   0.00-5.00   sec   738 MBytes  1.24 Gbits/sec  0.000 ms  0/529840 (0%)  sender
   [speedtest(4309)] [ 27]   0.00-5.00   sec   271 MBytes   454 Mbits/sec  0.000 ms  335130/529650 (63%)  receiver
   [speedtest(4309)] client(sender): bytes_recv=283999200, bytes_sent=773566400, sender_time=5.000, recver_time=5.000
   [speedtest(4309)] client(sender): up_speed:  454 Mbits/sec
   [speedtest(4309)]
   [speedtest(4309)] speed test Done.
   [speedtest(4309)] start downloading test. 
   [speedtest(4309)] Connecting to host 172.16.200.1, port 7000
   [speedtest(4309)] Reverse mode, remote host 172.16.200.1 is sending
   [speedtest(4309)] [ 27] local 172.16.200.4 port 19586 connected to 172.16.200.1 port 7000
   [speedtest(4309)] [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
   [speedtest(4309)] [ 27]   0.00-1.00   sec  56.1 MBytes   471 Mbits/sec  0.005 ms  70258/110574 (64%)
   [speedtest(4309)] [ 27]   1.00-2.00   sec  56.0 MBytes   470 Mbits/sec  0.006 ms  66496/106740 (62%)
   [speedtest(4309)] [ 27]   2.00-3.00   sec  56.0 MBytes   470 Mbits/sec  0.005 ms  66481/106736 (62%)
   [speedtest(4309)] [ 27]   3.00-4.00   sec  56.1 MBytes   471 Mbits/sec  0.007 ms  66403/106690 (62%)
   [speedtest(4309)] [ 27]   4.00-5.00   sec  56.3 MBytes   473 Mbits/sec  0.008 ms  65991/106454 (62%)
   [speedtest(4309)] [ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
   [speedtest(4309)] [ 27]   0.00-5.00   sec   749 MBytes  1.26 Gbits/sec  0.000 ms  0/538110 (0%)  sender
   [speedtest(4309)] [ 27]   0.00-5.00   sec   281 MBytes   471 Mbits/sec  0.008 ms  335629/537194 (62%)  receiver
   [speedtest(4309)] client(recver): bytes_recv=294284900, bytes_sent=785640600, sender_time=5.000, recver_time=5.001
   [speedtest(4309)] client(recver): down_speed:  471 Mbits/sec
   [speedtest(4309)]
   [speedtest(4309)] speed test Done.
   fcron_speedtest_notify_func()-1275: Speed test pid=4309 done
   
   fcron_speedtest_on_test_finish()-1211: Test 380011 for 'spoke21-p1' succeed with up=454398, down=470794
   fcron_speedtest_save_results()-1144: Write logs to disk: succ=1, fail=0
   fcron_speedtest_sync_results()-1172: Sync cached results to secondary devices.

3. After the speed test schedule runs, view the result on the hub (FGT_A):

   The server side uses speedtestd, while the client side uses speedtest.

   The speed test results are applied on hub-phase1_0 and hub_phase1_1 as egress traffic shaping.

   # diagnose debug application speedtestd -1
   
   ......
   [speedtest(2771)] [  7] local 172.16.200.1 port 7000 connected to 172.16.200.2 port 17553
   ......
   [speedtest(2771)] [  7] local 172.16.200.1 port 7000 connected to 172.16.200.2 port 7998
   ......
   [sptestd::ctrl(0377):root] set shaper: if=hub-phase1, tun=hub-phase1_0, sp=profile_1, bw=459745
   ......
   [speedtest(2771)] [  7] local 172.16.200.1 port 7000 connected to 172.16.200.4 port 15349
   ......
   [speedtest(2771)] [  7] local 172.16.200.1 port 7000 connected to 172.16.200.4 port 19586
   ......
   [sptestd::ctrl(0377):root] set shaper: if=hub-phase1, tun=hub-phase1_1, sp=profile_1, bw=470855 
   ......

4. Verify the result is cached on the spokes.

   • On FGT_B, the speed test results are cached:

     #  diagnose test application forticron 10
     Speed test results:
     1: vdom=root, phase1intf=spoke11-p1, peer-id='172.16.200.1', up=454043, dw=459694, time=12/13 12:32:19

   • On FGT_D, the speed test results are cached:

     #   diagnose test application forticron 10
     Speed test results:
     1: vdom=root, phase1intf=spoke21-p1, peer-id='172.16.200.1', up=454398, dw=470794, time=12/12 16:33:18

5. On the hub (FGT_A), verify the speed test results are applied to the hub's IPsec tunnels as egress traffic shaping:

   On hub-phase1_0 and hub-phase1_1, the correct traffic control is displayed.

   # diagnose vpn tunnel list
   list all ipsec tunnel in vd 0
   ......
   ------------------------------------------------------
   name=hub-phase1_0 ver=2 serial=16 172.16.200.1:0->172.16.200.2:0 tun_id=10.10.15.1 tun_id6=2000:10:10:15::1 dst_mtu=1500 dpd-link=on weight=1
   bound_if=11 lgwy=static/1 tun=intf mode=dial_inst/3 encap=none/74408 options[122a8]=npu rgwy-chg frag-rfc  run_state=0 role=primary accept_traffic=1 overlay_id=10
   
   parent=hub-phase1 index=0
   ......
   egress traffic control:
           bandwidth=459745(kbps) lock_hit=0 default_class=2 n_active_class=3
           class-id=2      allocated-bandwidth=45974(kbps)         guaranteed-bandwidth=45974(kbps)
                           max-bandwidth=45974(kbps)       current-bandwidth=0(kbps)
                           priority=low    forwarded_bytes=86K
                           dropped_packets=0       dropped_bytes=0
           class-id=3      allocated-bandwidth=137923(kbps)        guaranteed-bandwidth=137923(kbps)
                           max-bandwidth=183897(kbps)      current-bandwidth=0(kbps)
                           priority=medium         forwarded_bytes=0
                           dropped_packets=0       dropped_bytes=0
           class-id=4      allocated-bandwidth=275846(kbps)        guaranteed-bandwidth=91948(kbps)
                           max-bandwidth=275846(kbps)      current-bandwidth=0(kbps)
                           priority=high   forwarded_bytes=0
                           dropped_packets=0       dropped_bytes=0
   ------------------------------------------------------
   name=hub-phase1_1 ver=2 serial=17 172.16.200.1:0->172.16.200.4:0 tun_id=10.10.15.2 tun_id6=2000:10:10:15::2 dst_mtu=1500 dpd-link=on weight=1
   bound_if=11 lgwy=static/1 tun=intf mode=dial_inst/3 encap=none/74408 options[122a8]=npu rgwy-chg frag-rfc  run_state=0 role=primary accept_traffic=1 overlay_id=10
   
   parent=hub-phase1 index=1
   ......
   egress traffic control:
           bandwidth=470855(kbps) lock_hit=0 default_class=2 n_active_class=3
           class-id=2      allocated-bandwidth=47085(kbps)         guaranteed-bandwidth=47085(kbps)
                           max-bandwidth=47085(kbps)       current-bandwidth=0(kbps)
                           priority=low    forwarded_bytes=81K
                           dropped_packets=0       dropped_bytes=0
           class-id=3      allocated-bandwidth=141256(kbps)        guaranteed-bandwidth=141256(kbps)
                           max-bandwidth=188341(kbps)      current-bandwidth=0(kbps)
                           priority=medium         forwarded_bytes=0
                           dropped_packets=0       dropped_bytes=0
           class-id=4      allocated-bandwidth=282512(kbps)        guaranteed-bandwidth=94170(kbps)
                           max-bandwidth=282512(kbps)      current-bandwidth=0(kbps)
                           priority=high   forwarded_bytes=0
                           dropped_packets=0       dropped_bytes=0