ADVPN 2.0 enhancements

This information is also available in the FortiOS 7.6 Administration Guide: ADVPN 2.0 edge discovery and path management

ADVPN 2.0 operation has been enhanced for SD-WAN. ADVPN 2.0 edge discovery and path management was introduced in FortiOS 7.4.2. See ADVPN 2.0 edge discovery and path management.

The following enhancements have been added:

• The local spoke directly sends a shortcut-query to a remote spoke to trigger a shortcut after ADVPN 2.0 path management makes a path decision.

• ADVPN 2.0 path management can trigger multiple shortcuts for load balancing SD-WAN rules. Traffic can be load balanced over these multiple shortcuts to use as much of the available WAN bandwidth as possible without wasting idle links if they are healthy. The algorithm to calculate multiple shortcuts for the load balancing service will consider transport group and in-SLA status for both local and remote parent overlays.

• Spokes can automatically deactivate all shortcuts connecting to the same spoke when user traffic is not observed for a specified time interval. This is achieved by enabling a shared idle timeout setting in the IPsec VPN Phase 1 interface settings for associated overlays.

CLI

The following commands configure the shared idle timeout for overlays used by ADVPN. The only new command is set shared-idle-timeout.

config vpn ipsec phase1-interface
   edit <phase1-interface name>
      set idle-timeout {enable | disable}
      set shared-idle-timeout {enable | disable}
      set idle-timeoutinterval <integer> 
   next
end

set idle-timeout {enable | disable}	Enable/disable IPsec tunnel idle timeout (default = disable). Must be set to enable when shared-idle-timeout is enabled.
set shared-idle-timeout {enable | disable}	Enable/disable shared-idle-timeout on involved overlays (default = disable).
set idle-timeoutinterval <integer>	IPsec tunnel idle timeout, in minutes (5 - 43200, default = 5).

The previous SD-WAN CLI commands continue to be used for configuring ADVPN 2.0. See SD-WAN CLI configuration.

Example

This example relies on the same network topology used in the previous ADVPN 2.0 topic, except for Spoke 1 having a single SD-WAN Rule/Service using the load balancing strategy with SLA targets. For details, see Network Topology and Load balancing strategy with SLA targets.

Recall the network topology is as follows:

• Standard Hub-Spoke topology

• The topology has two spokes, and each spoke has three overlays to Hub. Two overlays are created on internet links, and another overlay is created on the MPLS link.

• BGP neighbor per overlay is established between spoke and hub.

SD-WAN configuration, selected IPsec configuration, and health check status

This section shows the SD-WAN configuration, selected IPsec configuration, and health check status on Spoke 1: and Spoke 2:.

Spoke 1:

config system sdwan
    set status enable
    config zone
        edit "virtual-wan-link"
        next
        edit "overlay"
            set advpn-select enable 
            set advpn-health-check "HUB"
        next
    end
    config members
        edit 1
            set interface "H1_T11"
            set zone "overlay"
            set transport-group 1 
        next
        edit 2
            set interface "H1_T22"
            set zone "overlay"
            set transport-group 1 
        next
        edit 3
            set interface "H1_T33"
            set zone "overlay"
            set transport-group 2 
        next
    end
    config health-check
        edit "HUB"
            set server "172.31.100.100"
            set members 1 2 3
            config sla
                edit 1
                    set link-cost-factor latency
                    set latency-threshold 100
                next
            end
        next
    end
    config service
        edit 1
            set name "1"
            set load-balance enable            
            set mode sla
            set dst "CORP_LAN"
            set src "CORP_LAN"
            config sla
                edit "HUB"
                    set id 1
                next
            end
            set priority-members 1 2 3
        next
    end
end
config vpn ipsec phase1-interface
    edit "H1_T11"
        ...
        set idle-timeout enable
        set shared-idle-timeout enable
        set idle-timeoutinterval 5
        ...
    next
end

config vpn ipsec phase1-interface
    edit "H1_T22"
        ...
        set idle-timeout enable
        set shared-idle-timeout enable
        set idle-timeoutinterval 5                            
        ...
    next
end

config vpn ipsec phase1-interface
    edit "H1_T33"
        ...
        set idle-timeout enable
        set shared-idle-timeout enable
        set idle-timeoutinterval 5
        ...
    next
end

# diagnose system sdwan  health-check 
Health Check(HUB):
Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.223), jitter(0.018), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1
Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.191), jitter(0.009), mos(4.404), bandwidth-up(999993), bandwidth-dw(999998), bandwidth-bi(1999991) sla_map=0x1
Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.139), jitter(0.007), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1

Spoke 2:

config system sdwan
    set status enable
    config zone
        edit "virtual-wan-link"
        next
        edit "overlay"
            set advpn-select enable 
            set advpn-health-check "HUB"
        next
    end
    config members
        edit 1
            set interface "H1_T11"
            set zone "overlay"
            set transport-group 1 
        next
        edit 2
            set interface "H1_T22"
            set zone "overlay"
            set transport-group 1 
        next
        edit 3
            set interface "H1_T33"
            set zone "overlay"
            set transport-group 2 
        next
    end
    config health-check
        edit "HUB"
            set server "172.31.100.100"
            set members 3 1 2
            config sla
                edit 1
                    set link-cost-factor latency
                    set latency-threshold 100
                next
            end
        next
    end
end
config vpn ipsec phase1-interface
    edit "H1_T11"
        ...
        set idle-timeout enable
        set shared-idle-timeout enable
        set idle-timeoutinterval 5
        ...
    next
end

config vpn ipsec phase1-interface
    edit "H1_T22"
        ...
        set idle-timeout enable
        set shared-idle-timeout enable
        set idle-timeoutinterval 5                            
        ...
    next
end

config vpn ipsec phase1-interface
    edit "H1_T33"
        ...
        set idle-timeout enable
        set shared-idle-timeout enable
        set idle-timeoutinterval 5
        ...
    next
end

# diagnose system sdwan health-check 
Health Check(HUB):
Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.148), jitter(0.021), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1
Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.183), jitter(0.010), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1
Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.163), jitter(0.005), mos(4.404), bandwidth-up(999994), bandwidth-dw(999998), bandwidth-bi(1999992) sla_map=0x1

Scenario: traffic matching SD-WAN rule 1

In this scenario, PC1 connected to Spoke 1 initiates an ICMP ping destined for PC1 connected to Spoke 2. Therefore, this user traffic matches SD-WAN rule 1 and triggers shortcut path selection and establishment.

On Spoke 1, in the IKE debug (diagnose debug application ike -1), debug messages indicate that multiple direct shortcut-query packets are being sent to Spoke 2:

ike :VWL_ADVPN_MSG_T_TRIGGER
ike V=root:0 looking up shortcut by addr 172.31.80.2, resp-name:H1_T11, name H1_T22, peer-addr 172.31.3.101:0
ike V=root:0:H1_T22: send shortcut-query
...
ike :VWL_ADVPN_MSG_T_TRIGGER
ike V=root:0 looking up shortcut by addr 172.31.81.2, resp-name:H1_T22, name H1_T22, peer-addr 172.31.3.105:0
ike V=root:0:H1_T22: send shortcut-query
...
ike :VWL_ADVPN_MSG_T_TRIGGER
ike V=root:0 looking up shortcut by addr 172.31.82.2, resp-name:H1_T33, name H1_T33, peer-addr 172.31.4.101:0
ike V=root:0:H1_T33: send shortcut-query
...

From the diagnostic command on Spoke 1, observe that multiple shortcuts are triggered in bold based on the ADVPN 2.0 path management calculation where in-SLA overlays within the same transport group were selected.

Branch1_FGT# diagnose system sdwan service4
Service(1): Address Mode(IPV4) flags=0x24200 use-shortcut-sla use-shortcut
 Tie break: cfg
 Shortcut priority: 3
  Gen(69), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(sla  hash-mode=round-robin)
  Member sub interface(8):
    1: seq_num(1), interface(H1_T11):
       1: H1_T11_0(103)
       2: H1_T11_1(104)
    2: seq_num(2), interface(H1_T22):
       1: H1_T22_0(105)
       2: H1_T22_1(106)
    3: seq_num(3), interface(H1_T33):
       1: H1_T33_0(100)
  Members(8):
    1: Seq_num(1 H1_T11 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    2: Seq_num(2 H1_T22 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    3: Seq_num(3 H1_T33 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    4: Seq_num(3 H1_T33_0 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    5: Seq_num(1 H1_T11_0 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    6: Seq_num(1 H1_T11_1 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    7: Seq_num(2 H1_T22_0 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    8: Seq_num(2 H1_T22_1 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
  Src address(1):
        10.0.0.0-10.255.255.255
  Dst address(1):
        10.0.0.0-10.255.255.255

From the diagnostic command on Spoke 2, observe the shortcuts in bold:

Branch2_FGT# diagnose system sdwan health-check
Health Check(HUB):
Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.137), jitter(0.021), mos(4.404), bandwidth-up(999999), bandwidth-dw(999999), bandwidth-bi(1999998) sla_map=0x1
Seq(3 H1_T33_0): state(alive), packet-loss(0.000%) latency(0.163), jitter(0.047), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1
Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.186), jitter(0.014), mos(4.404), bandwidth-up(999999), bandwidth-dw(999999), bandwidth-bi(1999998) sla_map=0x1
Seq(1 H1_T11_0): state(alive), packet-loss(0.000%) latency(0.262), jitter(0.032), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1Seq(1 H1_T11_1): state(alive), packet-loss(0.000%) latency(0.255), jitter(0.037), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1
Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.167), jitter(0.009), mos(4.404), bandwidth-up(999995), bandwidth-dw(999999), bandwidth-bi(1999994) sla_map=0x1
Seq(2 H1_T22_0): state(alive), packet-loss(0.000%) latency(0.232), jitter(0.013), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1Seq(2 H1_T22_1): state(alive), packet-loss(0.000%) latency(0.257), jitter(0.021), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1

At this point, PC1 connected to Spoke 1 initiated multiple ICMP pings destined for PC1 connected to Spoke 2. The packet capture diagnostic command on Spoke 1 demonstrates that these ICMP pings have been load balanced over all shortcuts:

Branch1_FGT# diagnose sniffer packet any 'host 10.0.4.2' 4
interfaces=[any]
filters=[host 10.0.4.2]
3.481994 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
3.482103 H1_T11_1 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
3.482799 H1_T11_1 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
3.482928 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply
4.614480 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
4.614580 H1_T33_0 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
4.615122 H1_T33_0 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
4.615152 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply
5.286394 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
5.286497 H1_T22_0 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
5.287129 H1_T22_0 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
5.287155 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply
6.079759 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
6.079883 H1_T22_1 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
6.080496 H1_T22_1 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
6.080537 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply
7.983357 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
7.983447 H1_T11_0 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
7.984078 H1_T11_0 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
7.984120 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply

Without user traffic traversing the shortcut during the idle interval time, from the diagnostic command on Spoke 1, observe that all shortcuts have been removed:

Branch1_FGT# diagnose system sdwan service4
Service(1): Address Mode(IPV4) flags=0x24200 use-shortcut-sla use-shortcut
 Tie break: cfg
 Shortcut priority: 3
  Gen(16), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(sla  hash-mode=round-robin)
  Members(3):
    1: Seq_num(1 H1_T11 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    2: Seq_num(2 H1_T22 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
    3: Seq_num(3 H1_T33 overlay), alive, sla(0x1), gid(2), num of pass(1), selected
  Src address(1):
        10.0.0.0-10.255.255.255
  Dst address(1):
        10.0.0.0-10.255.255.255