Example SD-WAN configurations using ADVPN 2.0
The configuration example illustrates the edge discovery and path management processes for a typical hub and spoke topology. This example focuses on SD-WAN configuration for steering traffic and establishing shortcuts in the direction from Spoke 1 to Spoke 2.
|
Currently, ADVPN 2.0 only supports IPv4. |
Network Topology
In this example, BGP per overlay was used for dynamic routing to distribute the LAN routes behind each spoke to the other spoke. However, this was a design choice. You can also use BGP on loopback for this example.
Spokes 1 and 2 have the following VPN overlays between themselves and the hub:
|
VPN Overlays |
IP address on Spoke 1 |
IP address on Spoke 2 |
|---|---|---|
|
H1_T11 |
172.31.80.1/32 |
172.31.80.2/32 |
|
H1_T22 |
172.31.81.1/32 |
172.31.81.2/32 |
|
H1_T33 |
172.31.82.1/32 |
172.31.82.2/32 |
SD-WAN Rules/Services defined on Spoke 1:
|
|
SD-WAN Rule/Service 1 |
SD-WAN Rule/Service 2 |
SD-WAN Rule/Service 3 |
|---|---|---|---|
|
|
H1_T11 |
H1_T22 |
H1_T33 |
|
H1_T22 |
H1_T11 |
H1_T11 |
|
|
H1_T33 |
H1_T33 |
H1_T22 |
|
|
Strategy for choosing outgoing interfaces |
Lowest cost (SLA) |
Lowest cost (SLA) |
Best quality, link cost factor: packet loss |
Throughout this example, transport group 1 is used for VPN overlays over Internet links while transport group 2 is used for the VPN overlay over an MPLS link.
In this example, user traffic is initiated behind Spoke 1 and destined to Spoke 2. Because of this, Spoke 1 is considered the local spoke, and Spoke 2 is considered the remote spoke.
SD-WAN configuration and health check status
SD-WAN configuration and health check status on Spoke 1:
config system sdwan
set status enable
config zone
edit "virtual-wan-link"
next
edit "overlay"
set advpn-select enable
set advpn-health-check "HUB"
next
end
config members
edit 1
set interface "H1_T11"
set zone "overlay"
set transport-group 1
next
edit 2
set interface "H1_T22"
set zone "overlay"
set transport-group 1
next
edit 3
set interface "H1_T33"
set zone "overlay"
set transport-group 2
next
end
config health-check
edit "HUB"
set server "172.31.100.100"
set members 1 2 3
config sla
edit 1
set link-cost-factor latency
set latency-threshold 100
next
end
next
end
config service
edit 1
set name "1"
set mode sla
set shortcut-priority enable
set dst "spoke-2_LAN-1" "Tunnel_IPs"
set src "spoke-1_LAN-1" "Tunnel_IPs"
config sla
edit "HUB"
set id 1
next
end
set priority-members 1 2 3
next
edit 2
set name "2"
set mode sla
set shortcut-priority enable
set dst "spoke-2_LAN-2" "Tunnel_IPs"
set src "spoke-1_LAN-1" "Tunnel_IPs"
config sla
edit "HUB"
set id 1
next
end
set priority-members 2 1 3
next
edit 3
set name "3"
set mode priority
set dst "spoke-2_LAN-3" "Tunnel_IPs"
set src "spoke-1_LAN-1" "Tunnel_IPs"
set health-check "HUB"
set link-cost-factor packet-loss
set priority-members 3 1 2
next
end
end
# diagnose sys sdwan health-check Health Check(HUB): Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.231), jitter(0.029), mos(4.404), bandwidth-up(999999), bandwidth-dw(999997), bandwidth-bi(1999996) sla_map=0x1 Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.193), jitter(0.010), mos(4.404), bandwidth-up(999994), bandwidth-dw(999997), bandwidth-bi(1999991) sla_map=0x1 Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.144), jitter(0.007), mos(4.404), bandwidth-up(999999), bandwidth-dw(999997), bandwidth-bi(1999996) sla_map=0x1
SD-WAN configuration and health check status on Spoke 2:
config system sdwan
set status enable
config zone
edit "virtual-wan-link"
next
edit "overlay"
set advpn-select enable
set advpn-health-check "HUB"
next
end
config members
edit 1
set interface "H1_T11"
set zone "overlay"
set cost 100
set transport-group 1
next
edit 2
set interface "H1_T22"
set zone "overlay"
set transport-group 1
next
edit 3
set interface "H1_T33"
set zone "overlay"
set transport-group 2
next
end
config health-check
edit "HUB"
set server "172.31.100.100"
set members 3 1 2
config sla
edit 1
set link-cost-factor latency
set latency-threshold 100
next
end
next
end
end
# diagnose sys sdwan health-check Health Check(HUB): Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.124), jitter(0.009), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1 Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.216), jitter(0.043), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1 Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.184), jitter(0.012), mos(4.404), bandwidth-up(999994), bandwidth-dw(999998), bandwidth-bi(1999992) sla_map=0x1
Scenario 1: Traffic matching SD-WAN rule 1
In this scenario, PC 1 connected to Spoke 1 initiates an ICMP ping destined for PC1 connected to Spoke 2. Therefore, this user traffic matches SD-WAN rule 1 and triggers shortcut path selection and establishment.
The Path Manager of Spoke 1 will calculate the best shortcut path by comparing transport group, link quality (for SLA mode), link cost, and member configuration order between Spoke 1 and Spoke 2.
For an SLA mode service, the following algorithm is followed for considering endpoints of the best shortcut path:
-
Overlays with the same transport group
-
In-SLA overlays
-
Lowest link cost overlays
-
Member configuration order as a final tiebreaker
Based on this algorithm, the Path Manager on Spoke 1 selects Spoke 1 H1_T11 because:
It is first in the priority-members order for SD-WAN rule 1, it has the lowest link cost, and it is within SLA. Likewise, the Path Manager on Spoke 1 selects Spoke 2 H1_T22 since it has the lowest link cost compared to Spoke 2 H1_T11 (which has a cost of 100), it is within SLA, and has the same transport group as Spoke 1 H1_T11. Therefore, the Path Manager of Spoke 1 calculates the best shortcut path as Spoke 1 H1_T11 to Spoke 2 H1_T22.
The Path Manager will advise IKE to establish the best shortcut and add it to SD-WAN rule 1 as follows:
Branch1_FGT# diagnose sys sdwan service
Service(1): Address Mode(IPV4) flags=0x4200 use-shortcut-sla use-shortcut
Tie break: cfg
Shortcut priority: 1
Gen(11), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(sla), sla-compare-order
Member sub interface(4):
2: seq_num(1), interface(H1_T11):
1: H1_T11_0(71)
Members(4):
1: Seq_num(1 H1_T11_0 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(1 H1_T11 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
3: Seq_num(2 H1_T22 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
4: Seq_num(3 H1_T33 overlay), alive, sla(0x1), gid(0), cfg_order(2), local cost(0), selected
Src address(2):
172.31.0.0-172.31.255.255
10.0.3.0-10.0.3.255
Dst address(2):
172.31.0.0-172.31.255.255
10.0.4.0-10.0.4.255
…
Since shortcut-priority is enabled, we observe that the shortcut is formed over the selected overlay path and prioritized over the parent overlay.
From the diagnostic command on Spoke 1, we observe the selected shortcut path in bold. (Note that the remote IP matches Spoke 2 H1_T22 in the corresponding table above.)
Branch1_FGT# diagnose sys sdwan advpn-session Session head(Branch2_FGT-0-overlay:1) (1) Service ID(1), last access(7809088), remote health check info(3) Selected path: local(H1_T11, port1) gw: 172.31.3.1 remote IP: 172.31.3.105(172.31.81.2) Remote information: 1: latency: 0.176267 jitter: 0.005733 pktloss: 0.000000 mos: 4.404302 sla: 0x1 cost: 0 transport_group: 1 bandwidth up: 999994 down: 999997 bidirection: 1999991 ipv4: 172.31.3.105(172.31.81.2) ipv6 2000:172:31:3::105(::) 2: latency: 0.119133 jitter: 0.004800 pktloss: 0.000000 mos: 4.404331 sla: 0x1 cost: 0 transport_group: 2 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.4.101(172.31.82.2) ipv6 1410:4b02::f088:93ee:7f00:0(c010:4b02::788a:93ee:7f00:0) 3: latency: 0.182400 jitter: 0.008800 pktloss: 0.000000 mos: 4.404295 sla: 0x1 cost: 100 transport_group: 1 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.3.101(172.31.80.2) ipv6 2000:172:31:3::101(d88a:93ee:7f00:0:d88a:93ee:7f00:0)
From the diagnostic command on Spoke 2, we observe the selected shortcut in bold.
Branch2_FGT# diagnose sys sdwan health-check Health Check(HUB): Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.122), jitter(0.004), mos(4.404), bandwidth-up(999999), bandwidth-dw(999997), bandwidth-bi(1999996) sla_map=0x1 Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.186), jitter(0.011), mos(4.404), bandwidth-up(999999), bandwidth-dw(999997), bandwidth-bi(1999996) sla_map=0x1 Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.180), jitter(0.005), mos(4.404), bandwidth-up(999994), bandwidth-dw(999997), bandwidth-bi(1999991) sla_map=0x1 Seq(2 H1_T22_0): state(alive), packet-loss(0.000%) latency(0.265), jitter(0.011), mos(4.404), bandwidth-up(999999), bandwidth-dw(999999), bandwidth-bi(1999998) sla_map=0x1
Scenario 2: Traffic matching SD-WAN rule 2
In this scenario, PC 1 connected to Spoke 1 initiates an ICMP ping destined for PC2 connected to Spoke 2. Therefore, this user traffic matches SD-WAN rule 2, and traffic will go through shortcut H1_T11_0 of Spoke 1 previously established in Scenario 1 above.
The local spoke generates local-out UDP packets and sends them to the hub to trigger an IKE shortcut message exchange with updated remote spoke WAN link information. The local spoke will receive this updated remote spoke WAN link information. Then the Path Manager of Spoke 1 will recalculate the best shortcut path by comparing transport group, link quality (for SLA mode), link cost, and member configuration order between Spoke 1 and Spoke 2.
For an SLA mode service, the following algorithm is followed for considering endpoints of the best shortcut path:
-
Overlays with the same transport group
-
In-SLA overlays
-
Lowest link cost overlays
-
Member configuration order as a final tiebreaker
Based on this algorithm, the Path Manager on Spoke 1 selects Spoke 1 H1_T22 because it is the first in the priority-members order for SD-WAN rule 2, it has the lowest link cost, and it is within SLA. Likewise, the Path Manager on Spoke 1 selects Spoke 2 H1_T22 since it has the lowest link cost compared to Spoke 2 H1_T11 (which has a cost of 100), it is within SLA, and has the same transport group as Spoke 1 H1_T11. Therefore, the Path Manager of Spoke 1 calculates the best shortcut path as Spoke 1 H1_T22 to Spoke 2 H1_T22.
The Path Manager will advise IKE to establish the best shortcut and add it to SD-WAN rule 2 as follows:
Branch1_FGT# diagnose sys sdwan service
…
Service(2): Address Mode(IPV4) flags=0x4200 use-shortcut-sla use-shortcut
Tie break: cfg
Shortcut priority: 1
Gen(12), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(sla), sla-compare-order
Member sub interface(5):
3: seq_num(2), interface(H1_T22):
1: H1_T22_0(72)
4: seq_num(1), interface(H1_T11):
1: H1_T11_0(71)
Members(5):
1: Seq_num(2 H1_T22_0 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(1 H1_T11_0 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected, last_used=2023-12-05 14:34:07
3: Seq_num(2 H1_T22 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
4: Seq_num(1 H1_T11 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
5: Seq_num(3 H1_T33 overlay), alive, sla(0x1), gid(0), cfg_order(2), local cost(0), selected
Src address(2):
172.31.0.0-172.31.255.255
10.0.3.0-10.0.3.255
Dst address(2):
172.31.0.0-172.31.255.255
10.0.40.0-10.0.40.255
…
The newly selected shortcut is prioritized over the previously selected shortcut as seen in the bolded output above.
From the diagnostic command on Spoke 1, we observe the selected shortcut path in bold. (Note that the remote IP matches Spoke 2 H1_T22 in the corresponding table above.)
Branch1_FGT# diagnose sys sdwan advpn-session Session head(Branch2_FGT-0-overlay:2) (1) Service ID(1), last access(8024725), remote health check info(3) Selected path: local(H1_T11, port1) gw: 172.31.3.1 remote IP: 172.31.3.105(172.31.81.2) Remote information: 1: latency: 0.118267 jitter: 0.004633 pktloss: 0.000000 mos: 4.404331 sla: 0x1 cost: 0 transport_group: 2 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.4.101(172.31.82.2) ipv6 180:adfb::d88a:93ee:7f00:0(d88a:93ee:7f00:0:d88a:93ee:7f00:0) 2: latency: 0.176067 jitter: 0.006567 pktloss: 0.000000 mos: 4.404301 sla: 0x1 cost: 0 transport_group: 1 bandwidth up: 999994 down: 999997 bidirection: 1999991 ipv4: 172.31.3.105(172.31.81.2) ipv6 2000:172:31:3::105(::) 3: latency: 0.170333 jitter: 0.008133 pktloss: 0.000000 mos: 4.404302 sla: 0x1 cost: 100 transport_group: 1 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.3.101(172.31.80.2) ipv6 2000:172:31:3::101(c010:4b02::788a:93ee:7f00:0) (1) Service ID(2), last access(8024725), remote health check info(3) Selected path: local(H1_T22, port2) gw: 172.31.3.5 remote IP: 172.31.3.105(172.31.81.2) Remote information: 1: latency: 0.118267 jitter: 0.004633 pktloss: 0.000000 mos: 4.404331 sla: 0x1 cost: 0 transport_group: 2 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.4.101(172.31.82.2) ipv6 180:adfb::d88a:93ee:7f00:0(d88a:93ee:7f00:0:d88a:93ee:7f00:0) 2: latency: 0.176067 jitter: 0.006567 pktloss: 0.000000 mos: 4.404301 sla: 0x1 cost: 0 transport_group: 1 bandwidth up: 999994 down: 999997 bidirection: 1999991 ipv4: 172.31.3.105(172.31.81.2) ipv6 2000:172:31:3::105(::) 3: latency: 0.170333 jitter: 0.008133 pktloss: 0.000000 mos: 4.404302 sla: 0x1 cost: 100 transport_group: 1 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.3.101(172.31.80.2) ipv6 2000:172:31:3::101(c010:4b02::788a:93ee:7f00:0) …
From the diagnostic command on Spoke 2, we observe the selected shortcut in bold:
Branch2_FGT# diagnose sys sdwan health-check Health Check(HUB): Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.118), jitter(0.005), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1 Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.171), jitter(0.005), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1 Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.175), jitter(0.006), mos(4.404), bandwidth-up(999994), bandwidth-dw(999998), bandwidth-bi(1999992) sla_map=0x1 Seq(2 H1_T22_0): state(alive), packet-loss(0.000%) latency(0.240), jitter(0.009), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1 Seq(2 H1_T22_1): state(alive), packet-loss(0.000%) latency(0.259), jitter(0.019), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1
Scenario 3: Traffic matching SD-WAN rule 3
In this scenario, PC 1 connected to Spoke 1 initiates an ICMP ping destined for PC 3 connected to Spoke 2. Therefore, this user traffic matches SD-WAN rule 3, and traffic will go through shortcut H1_T11_0 of Spoke 1 previously established in Scenario 1 above.
The local spoke generates local-out UDP packets and sends them to the hub to trigger an IKE shortcut message exchange with updated remote spoke WAN link information. The local spoke will receive this updated remote spoke WAN link information. Then the Path Manager of Spoke 1 will recalculate the best shortcut path by comparing transport group, best quality (based on link cost factor), and member configuration order between Spoke 1 and Spoke 2.
For a best quality mode service, the following algorithm is followed for considering endpoints of the best shortcut path:
-
Overlays with the same transport group
-
Best quality overlays (link cost factor of packet loss, in this scenario)
-
Member configuration order as a final tiebreaker
Based on this algorithm, the Path Manager on Spoke 1 selects Spoke 1 H1_T33 because it is the first in the priority-members order for SD-WAN rule 3, and it has the best quality link. Likewise, the Path Manager on Spoke 1 selects Spoke 2 H1_T33 since it has the same transport group as Spoke 1 H1_T33. Therefore, the Path Manager of Spoke 1 calculates the best shortcut path as Spoke 1 H1_T33 to Spoke 2 H1_T33.
The Path Manager will advise IKE to establish the best shortcut and add it to SD-WAN rule 3 as follows:
Branch1_FGT# diagnose sys sdwan service
…
Service(3): Address Mode(IPV4) flags=0x4200 use-shortcut-sla use-shortcut
Tie break: cfg
Shortcut priority: 3
Gen(13), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(priority), link-cost-factor(packet-loss), link-cost-threshold(10), heath-check(HUB)
Member sub interface(6):
4: seq_num(3), interface(H1_T33):
1: H1_T33_0(73)
5: seq_num(1), interface(H1_T11):
1: H1_T11_0(71)
6: seq_num(2), interface(H1_T22):
1: H1_T22_0(72)
Members(6):
1: Seq_num(3 H1_T33_0 overlay), alive, packet loss: 0.000%, selected
2: Seq_num(1 H1_T11_0 overlay), alive, packet loss: 0.000%, selected, last_used=2023-12-05 14:38:02
3: Seq_num(2 H1_T22_0 overlay), alive, packet loss: 0.000%, selected
4: Seq_num(3 H1_T33 overlay), alive, packet loss: 0.000%, selected
5: Seq_num(1 H1_T11 overlay), alive, packet loss: 0.000%, selected
6: Seq_num(2 H1_T22 overlay), alive, packet loss: 0.000%, selected
Src address(2):
172.31.0.0-172.31.255.255
10.0.3.0-10.0.3.255
Dst address(2):
172.31.0.0-172.31.255.255
10.0.41.0-10.0.41.255
From the diagnostic command on Spoke 1, we observe the selected shortcut path in bold. (Note that the remote IP matches Spoke 2 H1_T33 in the corresponding table above.)
Branch1_FGT# diagnose sys sdwan advpn-session Session head(Branch2_FGT-0-overlay:3) (1) Service ID(3), last access(8047297), remote health check info(3) Selected path: local(H1_T33, port3) gw: 172.31.4.1 remote IP: 172.31.4.101(172.31.82.2) Remote information: 1: latency: 0.116600 jitter: 0.004600 pktloss: 0.000000 mos: 4.404332 sla: 0x1 cost: 0 transport_group: 2 bandwidth up: 999999 down: 999998 bidirection: 1999997 ipv4: 172.31.4.101(172.31.82.2) ipv6 180:adfb::d88a:93ee:7f00:0(d88a:93ee:7f00:0:d88a:93ee:7f00:0) 2: latency: 0.174767 jitter: 0.005533 pktloss: 0.000000 mos: 4.404303 sla: 0x1 cost: 0 transport_group: 1 bandwidth up: 999994 down: 999998 bidirection: 1999992 ipv4: 172.31.3.105(172.31.81.2) ipv6 2000:172:31:3::105(c010:4b02::788a:93ee:7f00:0) 3: latency: 0.172900 jitter: 0.005167 pktloss: 0.000000 mos: 4.404304 sla: 0x1 cost: 100 transport_group: 1 bandwidth up: 999999 down: 999998 bidirection: 1999997 ipv4: 172.31.3.101(172.31.80.2) ipv6 2000:172:31:3::101(::)
From the diagnostic command on Spoke 2, we observe the selected shortcut in bold:
Branch2_FGT# diagnose sys sdwan health-check Health Check(HUB): Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.116), jitter(0.005), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1 Seq(3 H1_T33_0): state(alive), packet-loss(0.000%) latency(0.113), jitter(0.005), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1 Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.171), jitter(0.004), mos(4.404), bandwidth-up(999999), bandwidth-dw(999998), bandwidth-bi(1999997) sla_map=0x1 Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(0.174), jitter(0.008), mos(4.404), bandwidth-up(999994), bandwidth-dw(999998), bandwidth-bi(1999992) sla_map=0x1 Seq(2 H1_T22_0): state(alive), packet-loss(0.000%) latency(0.239), jitter(0.007), mos(4.404), bandwidth-up(999999), bandwidth-dw(999999), bandwidth-bi(1999998) sla_map=0x1 Seq(2 H1_T22_1): state(alive), packet-loss(0.000%) latency(0.260), jitter(0.014), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1
Scenario 4: Spoke 2 H1_T22 overlay link out-of-SLA
In this scenario, we place remote Spoke 2 H1_T22 out-of-SLA and observe that this link quality change is sensed by the local spoke through regular WAN link information updates on shortcuts. Then the local Spoke 1 will generate local-out UDP packets and send them to the hub to trigger an IKE shortcut message exchange. Once Spoke 1 receives a shortcut reply, it will start to calculate new best shortcut paths for SD-WAN rules 1 and 2 because these are the only rules that have new best shortcut paths when Spoke 2 H1_T22 is out-of-SLA.
For an SLA mode service, the following algorithm is followed for considering endpoints of the best shortcut path:
-
Overlays with the same transport group
-
In-SLA overlays
-
Lowest link cost overlays
-
Member configuration order as a final tiebreaker
Based on this algorithm, the Path Manager on Spoke 1 still selects these Spoke 1 interfaces:
-
SD-WAN Rule 1: H1_T11
-
SD-WAN Rule 2: H1_T22
These are the first in the priority-members order for SD-WAN rules 1 and 2, respectively.
Based on the updated WAN link information, the Path Manager on Spoke 1 selects these Spoke 2 interfaces because they are the only remaining in-SLA VPN overlays over Internet links (transport group 1):
-
SD-WAN Rule 1: H1_T11
-
SD-WAN Rule 2: H1_T11
Therefore, the Path Manager of Spoke 1 calculates the best shortcut paths as follows:
-
SD-WAN Rule 1: Spoke 1 H1_T11 to Spoke 2 H1_T11
-
SD-WAN Rule 2: Spoke 1 H1_T22 to Spoke 2 H1_T11
The Path Manager will advise IKE to establish the best shortcuts and add them to SD-WAN rules 1 and 2 as follows:
-
For SD-WAN Rule 1, H1_T11_1 is the new best shortcut.
-
For SD-WAN Rule 2, H1_T22_1 is the new best shortcut.
# diagnose sys sdwan service
Service(1): Address Mode(IPV4) flags=0x4200 use-shortcut-sla use-shortcut
Tie break: cfg
Shortcut priority: 1
Gen(17), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(sla), sla-compare-order
Member sub interface(8):
6: seq_num(1), interface(H1_T11):
1: H1_T11_0(74)
2: H1_T11_1(75)
7: seq_num(2), interface(H1_T22):
1: H1_T22_0(72)
2: H1_T22_1(76)
8: seq_num(3), interface(H1_T33):
1: H1_T33_0(73)
Members(8):
1: Seq_num(1 H1_T11_0 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(1 H1_T11_1 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
3: Seq_num(2 H1_T22_0 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
4: Seq_num(2 H1_T22_1 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
5: Seq_num(3 H1_T33_0 overlay), alive, sla(0x1), gid(0), cfg_order(2), local cost(0), selected
6: Seq_num(1 H1_T11 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
7: Seq_num(2 H1_T22 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
8: Seq_num(3 H1_T33 overlay), alive, sla(0x1), gid(0), cfg_order(2), local cost(0), selected
Src address(2):
172.31.0.0-172.31.255.255
10.0.3.0-10.0.3.255
Dst address(2):
172.31.0.0-172.31.255.255
10.0.4.0-10.0.4.255
Service(2): Address Mode(IPV4) flags=0x4200 use-shortcut-sla use-shortcut
Tie break: cfg
Shortcut priority: 1
Gen(17), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(sla), sla-compare-order
Member sub interface(8):
6: seq_num(2), interface(H1_T22):
1: H1_T22_0(72)
2: H1_T22_1(76)
7: seq_num(1), interface(H1_T11):
1: H1_T11_0(74)
2: H1_T11_1(75)
8: seq_num(3), interface(H1_T33):
1: H1_T33_0(73)
Members(8):
1: Seq_num(2 H1_T22_0 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(2 H1_T22_1 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected3: Seq_num(1 H1_T11_1 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
4: Seq_num(1 H1_T11_0 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
5: Seq_num(3 H1_T33_0 overlay), alive, sla(0x1), gid(0), cfg_order(2), local cost(0), selected
6: Seq_num(2 H1_T22 overlay), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
7: Seq_num(1 H1_T11 overlay), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
8: Seq_num(3 H1_T33 overlay), alive, sla(0x1), gid(0), cfg_order(2), local cost(0), selected
Src address(2):
172.31.0.0-172.31.255.255
10.0.3.0-10.0.3.255
Dst address(2):
172.31.0.0-172.31.255.255
10.0.40.0-10.0.40.255
…
From the diagnostic command on Spoke 1, we observe the newly selected shortcut paths in bold. (Note that the remote IP 172.31.80.2 matches Spoke 2 H1_T11, which is the VPN overlay over the Internet link with cost 100 in the corresponding table above.)
# diagnose sys sdwan advpn-session Session head(Branch2_FGT-0-overlay:3) (1) Service ID(1), last access(8293060), remote health check info(3) Selected path: local(H1_T11, port1) gw: 172.31.3.1 remote IP: 172.31.3.101(172.31.80.2) Remote information: 1: latency: 0.119500 jitter: 0.006067 pktloss: 0.000000 mos: 4.404329 sla: 0x1 cost: 0 transport_group: 2 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.4.101(172.31.82.2) ipv6 180:adfb::d88a:93ee:7f00:0(d88a:93ee:7f00:0:d88a:93ee:7f00:0) 2: latency: 250.170761 jitter: 0.011500 pktloss: 0.000000 mos: 3.992655 sla: 0x0 cost: 0 transport_group: 1 bandwidth up: 999994 down: 999997 bidirection: 1999991 ipv4: 172.31.3.105(172.31.81.2) ipv6 2000:172:31:3::105(c010:4b02::788a:93ee:7f00:0) 3: latency: 0.182200 jitter: 0.012000 pktloss: 0.000000 mos: 4.404292 sla: 0x1 cost: 100 transport_group: 1 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.3.101(172.31.80.2) ipv6 2000:172:31:3::101(::) (1) Service ID(2), last access(8293060), remote health check info(3) Selected path: local(H1_T22, port2) gw: 172.31.3.5 remote IP: 172.31.3.101(172.31.80.2) Remote information: 1: latency: 0.119500 jitter: 0.006067 pktloss: 0.000000 mos: 4.404329 sla: 0x1 cost: 0 transport_group: 2 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.4.101(172.31.82.2) ipv6 180:adfb::d88a:93ee:7f00:0(d88a:93ee:7f00:0:d88a:93ee:7f00:0) 2: latency: 250.170761 jitter: 0.011500 pktloss: 0.000000 mos: 3.992655 sla: 0x0 cost: 0 transport_group: 1 bandwidth up: 999994 down: 999997 bidirection: 1999991 ipv4: 172.31.3.105(172.31.81.2) ipv6 2000:172:31:3::105(c010:4b02::788a:93ee:7f00:0) 3: latency: 0.182200 jitter: 0.012000 pktloss: 0.000000 mos: 4.404292 sla: 0x1 cost: 100 transport_group: 1 bandwidth up: 999999 down: 999997 bidirection: 1999996 ipv4: 172.31.3.101(172.31.80.2) ipv6 2000:172:31:3::101(::)
From the diagnostic command on Spoke 2, we observe the selected shortcuts in bold:
Branch2_FGT# diagnose sys sdwan health-check Health Check(HUB): Seq(3 H1_T33): state(alive), packet-loss(0.000%) latency(0.120), jitter(0.007), mos(4.404), bandwidth-up(999999), bandwidth-dw(999997), bandwidth-bi(1999996) sla_map=0x1 Seq(3 H1_T33_0): state(alive), packet-loss(0.000%) latency(0.128), jitter(0.003), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1 Seq(1 H1_T11): state(alive), packet-loss(0.000%) latency(0.180), jitter(0.008), mos(4.404), bandwidth-up(999999), bandwidth-dw(999997), bandwidth-bi(1999996) sla_map=0x1 Seq(1 H1_T11_0): state(alive), packet-loss(0.000%) latency(0.259), jitter(0.023), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1 Seq(1 H1_T11_1): state(alive), packet-loss(0.000%) latency(0.257), jitter(0.014), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1 Seq(2 H1_T22): state(alive), packet-loss(0.000%) latency(250.169), jitter(0.009), mos(3.993), bandwidth-up(999994), bandwidth-dw(999997), bandwidth-bi(1999991) sla_map=0x0 Seq(2 H1_T22_1): state(alive), packet-loss(0.000%) latency(0.245), jitter(0.013), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1 Seq(2 H1_T22_0): state(alive), packet-loss(0.000%) latency(0.223), jitter(0.005), mos(4.404), bandwidth-up(1000000), bandwidth-dw(1000000), bandwidth-bi(2000000) sla_map=0x1