Spoke learns egress shaping value from other spoke over ADVPN shortcut

With peer egress shaping for IPsec tunnels enabled, IKE bandwidth negotiation is used for traffic shaping. This configuration is useful for speed tests of large-scale SD-WAN deployments with thousands of sites using diverse connections (4G/5G, satellite, broadband).

ADVPN spoke-to-spoke shaping evaluates both the local and remote peers' egress shaping values and applies the lower of the two to optimize traffic flow. Additionally, this update enables the synchronization of the peer-egress-shaping-value during ADVPN shortcut reply messages sent to the shortcut initiator. Subsequently, during the regular IPsec negotiation, the initiator shares its shaping value with the responder, allowing shaping to be applied accordingly.

config vpn ipsec phase1-interface
    edit <tunnel_name>
        set ike-version 2
        set peer-egress-shaping {enable | disable}
        set peer-egress-shaping-value <integer>  
    next
end

Command	Description
peer-egress-shaping {enable | disable}	Enable/disable peer egress shaping.
peer-egress-shaping-value <integer>	Configure outbound bandwidth to use for peer egress shaping, in kbps (0 - 80000000, default = 0).

Example

This example shows the configuration of a hub with two spokes.

A peer-egress-shaping-value is configured on both Spoke 1 and Spoke 2, and the peer spoke learns the value through ADVPN's shortcut-reply. Subsequently, each spoke applies the egress bandwidth limit to their overlay tunnel.

The IP addresses are as follows:

FortiGate	Interface	IP adress
Spoke1	Port1	172.16.200.1
Spoke1 PC	Port2	10.1.100.1
Spoke2	Port1	172.16.200.3
Spoke2 PC	Port2	192.168.4.3
Hub	Port1	172.16.200.4
Hub PC	Port2	192.168.5.4

The configuration steps are:

1. Configure the Hub:

   1. Configure the IPsec VPN

   2. Configure the static route

   3. Configure the firewall policies

2. Configure Spoke1:

   1. Configure the IPsec VPN

   2. Configure the static route

   3. Configure the firewall policies

   4. Configure the shaping class

   5. Configure the shaping profile

   6. Apply shaping profile to Spoke1’s tunnel interface

3. Configure Spoke2:

   1. Configure the IPsec VPN

   2. Configure the static route

   3. Configure the firewall policies

   4. Configure the shaping class

   5. Configure the shaping profile

   6. Apply shaping profile to Spoke2’s tunnel interface

To configure the hub:

1. Configure the IPsec VPN:

   config vpn ipsec phase1-interface
       edit "tospokes"
           set type dynamic
           set interface "port1"
           set ike-version 2
           set peertype any
           set net-device disable
           set mode-cfg enable
           set proposal aes128-sha256 aes256-sha256 aes128gcm-prfsha256 aes256gcm-prfsha384 chacha20poly1305-prfsha256
           set add-route disable
           set dpd on-idle
           set npu-offload disable
           set auto-discovery-sender enable
           set transport auto
           set ipv4-start-ip 100.100.100.10
           set ipv4-end-ip 100.100.100.20
           set ipv4-netmask 255.255.255.0
           set psksecret ******
           set dpd-retryinterval 60
       next
   end

   config vpn ipsec phase2-interface
       edit "tospokes"
           set phase1name "tospokes"
           set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
       next
   end
   

2. Configure the static route:

   config router static
       edit 1
           set device "tospokes"
       next
   end
   

3. Configure the firewall policies:

   config firewall policy
       edit 1
           set srcintf "port2"
           set dstintf "tospokes"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
       edit 2
           set srcintf "tospokes"
           set dstintf "port2"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
       edit 3
           set srcintf "tospokes"
           set dstintf "tospokes"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
   end
   

To configure spoke1:

1. Configure the IPsec VPN:

   config vpn ipsec phase1-interface
       edit "tohub-d"
           set interface "port1"
           set ike-version 2
           set peertype any
           set net-device disable
           set peer-egress-shaping enable                
           set peer-egress-shaping-value 500             
           set mode-cfg enable
           set proposal aes128-sha256 aes256-sha256 aes128gcm-prfsha256 aes256gcm-prfsha384 chacha20poly1305-prfsha256
           set add-route disable
           set auto-discovery-shortcuts enable
           set auto-discovery-receiver enable
           set transport auto
           set remote-gw 172.16.200.4
           set psksecret ******
       next
   end

   config vpn ipsec phase2-interface
       edit "tohub-d"
           set phase1name "tohub-d"
           set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
           set auto-negotiate enable
       next
   end
   

2. Configure the static route:

   config router static
       edit 1
           set device "tohub-d"
       next
   end
   

3. Configure the firewall policies:

   config firewall policy
       edit 1
           set srcintf "port2"
           set dstintf "tohub-d"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
       edit 2
           set srcintf "tohub-d"
           set dstintf "port2"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
   end
   

4. Configure the shaping class:

   config firewall traffic-class
       edit 2
           set class-name "class_id_2"
       next
   end
   

5. Configure the shaping profile:

   config firewall shaping-profile
       edit "spoke_spoke"
           set type policing
           set default-class-id 2
           config shaping-entries
               edit 1
                   set class-id 2
                   set guaranteed-bandwidth-percentage 100
                   set maximum-bandwidth-percentage 100
               next
           end
       next
   end
   

6. Apply shaping profile to Spoke1’s tunnel interface:

   config system interface
       edit "tospokes"
           set egress-shaping-profile "hub_spoke"
       next
   end
   

   When configuring egress shaping on a tunnel interface, do not manually set the outbandwidth value. This value is dynamically obtained from the PEER_EGRESS_SHAPING_VALUE for each tunnel and should be left at its default value of 0.

To configure spoke2:

1. Configure the IPsec VPN:

   config vpn ipsec phase1-interface
       edit "tohub-d"
           set interface "port1"
           set ike-version 2
           set peertype any
           set net-device disable
           set peer-egress-shaping enable           
           set peer-egress-shaping-value 500        
           set mode-cfg enable
           set proposal aes128-sha256 aes256-sha256 aes128gcm-prfsha256 aes256gcm-prfsha384 chacha20poly1305-prfsha256
           set add-route disable
           set auto-discovery-receiver enable
           set auto-discovery-shortcuts dependent
           set transport auto
           set remote-gw 172.16.200.4
           set psksecret ******
       next
   end

   config vpn ipsec phase2-interface
       edit "tohub-d"
           set phase1name "tohub-d"
           set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
           set auto-negotiate enable
       next
   end
   

2. Configure the static route:

   config router static
       edit 1
           set device "tohub-d"
       next
   end
   

3. Configure the firewall policies:

   config firewall policy
       edit 1
           set srcintf "port2"
           set dstintf "tohub-d"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
       edit 2
           set srcintf "tohub-d"
           set dstintf "port2"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
   end
   

4. Configure the shaping class:

   config firewall traffic-class
       edit 2
           set class-name "class_id_2"
       next
   end
   

5. Configure the shaping profile:

   config firewall shaping-profile
       edit "spoke_spoke"
           set type policing
           set default-class-id 2
           config shaping-entries
               edit 1
                   set class-id 2
                   set guaranteed-bandwidth-percentage 100
                   set maximum-bandwidth-percentage 100
               next
           end
       next
   end
   

6. Apply shaping profile to Spoke2’s tunnel interface:

   config system interface
       edit "tospokes"
           set egress-shaping-profile "hub_spoke"
       next
   end
   

To verify the configuration:

1. Verify that traffic can be send from Spoke1 to Spoke2:

   # ping 192.168.4.3
   Pinging 192.168.4.3 with 32 bytes of data:
   Reply from 192.168.4.3: bytes=32 time=2ms TTL=62
   Reply from 192.168.4.3: bytes=32 time=2ms TTL=62
   Reply from 192.168.4.3: bytes=32 time=1ms TTL=62
   Reply from 192.168.4.3: bytes=32 time<1ms TTL=62
   
   Ping statistics for 192.168.4.3:
       Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
   Approximate round trip times in milli-seconds:
       Minimum = 0ms, Maximum = 2ms, Average = 1ms
   

2. Check Spoke1 tunnel shaping:

   # diagnose vpn tunnel list | grep -A 5 "egress traffic control"
   egress traffic control:
           bandwidth=500(kbps) lock_hit=1 default_class=2 n_active_class=1
           class-id=2      allocated-bandwidth=500(kbps)   guaranteed-bandwidth=500(kbps)
                           max-bandwidth=500(kbps)         current-bandwidth=1(kbps)
                           priority=high   forwarded_bytes=84
                           dropped_packets=0       dropped_bytes=0 
   

3. Check Spoke2 tunnel shaping:

   # diagnose vpn tunnel list | grep -A 5 "egress traffic control"
   egress traffic control:
           bandwidth=500(kbps) lock_hit=0 default_class=2 n_active_class=1
           class-id=2      allocated-bandwidth=500(kbps)   guaranteed-bandwidth=500(kbps)
                           max-bandwidth=500(kbps)         current-bandwidth=1(kbps)
                           priority=high   forwarded_bytes=824
                           dropped_packets=0       dropped_bytes=0 
   

4. Confirm IKE negotiation on Spoke1:

   # diagnose debug application ike -1
   ike V=root:0:tohub-d:7: received informational request 
   ike V=root:0:tohub-d:7: processing notify type SHORTCUT_REPLY 
   ike V=root:0:tohub-d: recv shortcut-reply 3150081142027665143 6cb78f05ef0079bc/11b6b9160e729839 172.16.200.3 to 10.1.100.11 0 psk 64 ppk 0 ver 2 mode 0 ext-mapping 172.16.200.3:0, network-id 0/0 shaping=500 
   ike V=root:0:tohub-d: iif 48 192.168.4.33->10.1.100.11 0 route lookup oif 8 port2 gwy 0.0.0.0 
   ike V=root:0:tohub-d: shortcut-reply received from 172.16.200.3:500, local-nat=no, peer-nat=no 
   ike V=root:0:tohub-d:tohub-d: created connection: 0x55c5c21d3860 7 172.16.200.1->172.16.200.3:500. ike V=root:0:tohub-d: adding new dynamic tunnel for 172.16.200.3:500 
   ike V=root:0:tohub-d_0: tunnel created tun_id 172.16.200.3/::172.16.200.3 remote_location 0.0.0.0 
   ike V=root:0:tohub-d_0: added new dynamic tunnel for 172.16.200.3:500 
   ike V=root:0:tohub-d_0: shortcut selector added new 1 a203
   ...
   ike V=root:0:tohub-d_0:10: initiator received AUTH msg 
   ike V=root:0:tohub-d_0:10: peer identifier IPV4_ADDR 172.16.200.3 
   ike V=root:0:tohub-d_0:10: auth verify done 
   ike V=root:0:tohub-d_0:10: initiator AUTH continuation 
   ike V=root:0:tohub-d_0:10: authentication succeeded 
   ike V=root:0:tohub-d_0:10: processing notify type MESSAGE_ID_SYNC_SUPPORTED 
   ike V=root:0:tohub-d_0:10: processing notify type INTERFACE_ADDR4 
   ike V=root:0:tohub-d_0:10: INTERFACE-ADDR4 100.100.100.11 
   ike V=root:0:tohub-d_0:10: processing notify type PEER_EGRESS_SHAPING_VALUE 
   ike V=root:0:tohub-d_0:10: received peer egress shaping value=500 
   ike V=root:0:tohub-d_0:10: established IKE SA 6cb78f05ef0079bc/11b6b9160e729839 
   ike V=root:0:tohub-d_0:10: auto-discovery receiver 
   ike V=root:0:tohub-d_0:10: auto-discovery 2 
   ike V=root:0:tohub-d_0:10: check peer route: if_addr4_rcvd=1, if_addr6_rcvd=0, mode_cfg=0 
   ike V=root:0:tohub-d_0:10: update peer route 0.0.0.0 -> 100.100.100.11 
   ike V=root:0:tohub-d_0: add peer static route 100.100.100.11 via 172.16.200.3 dev tohub-d_0 
   ike V=root:0:tohub-d: schedule auto-negotiate
   ...   
   

5. Confirm IKE negotiation on Spoke2:

   # diagnose debug application ike -1 
   ike V=root:0:tohub-d:7: processing notify type SHORTCUT_QUERY 
   ike V=root:0:tohub-d: recv shortcut-query 3150081142027665143 6cb78f05ef0079bc/0000000000000000 172.16.200.1 10.1.100.11:2048->192.168.4.33:0 0 psk 64 ppk 0 ttl 31 nat 0 ver 2 mode 0 network-id 0 ike V=root:0:tohub-d: iif 46 10.1.100.11->192.168.4.33 0 route lookup oif 8 port2 gwy 0.0.0.0 ike V=root:0:tohub-d: shortcut-query received from 172.16.200.1:500, local-nat=no, peer-nat=no 
   ike 0:6cb78f05ef0079bc/11b6b9160e729839:8: out FF 
   ike V=root:0:ff00000000000000/2100000000000000:f32a0000:8: sent 
   IKE msg (keepalive): 172.16.200.3:500->172.16.200.1:500, len=1, vrf=0, id=ff00000000000000/2100000000000000:f32a0000, oif=7 
   ike V=root:0:tohub-d: send vwl advpn-info request (0x763f04e2) for intf tohub-d ike V=root:0:tohub-d: send vwl request error: No such file or directory 
   ike V=root:0:tohub-d: send shortcut-reply 3150081142027665143 6cb78f05ef0079bc/11b6b9160e729839 172.16.200.3 to 10.1.100.11 0 psk 64 ppk 0 ver 2 mode 0 network-id 0/0 shaping=500
   …
   ike V=root:0:tohub-d:8: responder received AUTH msg 
   ike V=root:0:tohub-d:8: processing notify type INITIAL_CONTACT 
   ike V=root:0:tohub-d:8: processing notify type INTERFACE_ADDR4 
   ike V=root:0:tohub-d:8: INTERFACE-ADDR4 100.100.100.10 
   ike V=root:0:tohub-d:8: processing notify type MESSAGE_ID_SYNC_SUPPORTED 
   ike V=root:0:tohub-d:8: processing notify type PEER_EGRESS_SHAPING_VALUE 
   ike V=root:0:tohub-d:8: received peer egress shaping value=500 
   ike V=root:0:tohub-d:8: peer identifier IPV4_ADDR 172.16.200.1 
   ike V=root:0:tohub-d:8: auth verify done 
   ike V=root:0:tohub-d:8: responder AUTH continuation 
   ike V=root:0:tohub-d:8: authentication succeeded 
   ike V=root:0:tohub-d:8: responder creating new child
   …