Enhanced BGP next hop updates and ADVPN shortcut override 7.0.4

To increase flexibility when controlling how BGP route's next hops are resolved, the tag-match mode can be configured:

config router bgp
  set tag-resolve-mode {disable | preferred | merge}
end

Best-match (disable)	Resolve the BGP route's next hops with best-matched routes. This is the default setting.
Tag-match (preferred)	Resolve the BGP route's next hops with routes that have the same tag. If there are no results, resolve the next hops with best-matched routes.
Tag-and-best-match (merge)	Merge tag-match with best-match if they are using different routes, then let shortcuts hide their parents. The results exclude the next hops of tag-match whose interfaces have appeared in best-match.

In these examples:

• Each spoke has two IPsec tunnels to each hub, and one BGP peer on loopback interface to each hub (route-reflector).

• The loopbacks are exchanged with IKE between the spokes and hubs. They are installed as static routes that are used to provide reachability for establishing BGP neighbors.

• The summary BGP routes from the loopback IP address ranges that originated on the hubs are advertised to the spokes for resolving the BGP next hop s on the spokes.

• The spokes' PC LAN subnets are reflected by the hubs.

• Spoke_1 receives BGP routes (the LAN subnet and loopback IP summary) from Hub_1 with tag 1 and from Hub_2 with tag 2.

• SD-WAN is enabled on Spoke_1, and all of the tunnels are SD-WAN members.

Example 1: Connection between Hub and Spoke down

If the connections between Hub_1 and Spoke_2 are down, traffic from PC_3 to PC_4 can still go through Hub_1 because of the best-match resolving on Spoke_1, but packets will be dropped on Hub_1. When tag-match is enabled on Spoke_1, the spoke will resolve the PC_4 LAN route to Hub2, and traffic will be forwarded to Hub_2 and reach its destination.

To test the tag-match mode:

1. View the key routes on Spoke_1:

   Spoke_1(root) # get router info routing-table all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 [2] (recursive via H1_T11 tunnel 172.31.1.1), 20:09:52
                 (recursive via H1_T22 tunnel 10.0.0.2), 20:09:52
                 (recursive via H2_T11 tunnel 172.31.1.101), 20:09:52
                 (recursive via H2_T22 tunnel 10.0.0.4), 20:09:52
   B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 23:25:37
                 (recursive via H1_T22 tunnel 10.0.0.2), 23:25:37
               [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 23:25:37
                 (recursive via H2_T22 tunnel 10.0.0.4), 23:25:37
   S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
               [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
   S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
               [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
   C       172.31.0.65/32 is directly connected, Loopback0
   ...

   172.31.0.0/25 is the loopback IP summary originated by both Hub_1 and Hub_2. The next hop of the PC_4 LAN route is resolved to Hub_1 (H1_T11, H1_T22) and Hub_2 (H2_T11, H2_T22) based on the loopback IP summary route.

2. When connections between Spoke_2 and Hub_1 fails due to the BGP neighbor, tunnels, or physical ports going down, the PC_4 LAN route can be still resolved to Hub_1 and Hub_2 because the loopback IP summary can still be received from both Hub_1 and Hub_2:

   Spoke_1(root) # get router info routing-table all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 (recursive via H1_T11 tunnel 172.31.1.1), 00:03:06
                 (recursive via H1_T22 tunnel 10.0.0.2), 00:03:06
                 (recursive via H2_T11 tunnel 172.31.1.101), 00:03:06
                 (recursive via H2_T22 tunnel 10.0.0.4), 00:03:06
   B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 23:55:34
                 (recursive via H1_T22 tunnel 10.0.0.2), 23:55:34
               [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 23:55:34
                 (recursive via H2_T22 tunnel 10.0.0.4), 23:55:34
   ...

3. If traffic sent from PC_3 to PC_4 goes through Hub_1, packets are dropped because there is no PC_4 LAN route on Hub_1:

   Spoke_1 (root) # diagnose sniffer packet any 'host 10.0.4.2' 4
   interfaces=[any]
   filters=[host 10.0.4.2]
   11.261264 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
   11.261349 H1_T11 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
   12.260268 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
   12.260291 H1_T11 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
   
   Hub_1 (root) #  diagnose sniffer packet any 'host 10.0.4.2' 4
   interfaces=[any]
   filters=[host 10.0.4.2]
   6.966064 EDGE_T1 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
   7.965012 EDGE_T1 in 10.0.3.2 -> 10.0.4.2: icmp: echo request

4. If the tag-match mode is set to tag-match (preferred) on Spoke_1, then the PC_4 LAN route can only be resolved to Hub_2 because of tag-match checking:

   Spoke_1(root) # get router info routing-table all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 00:02:35
                 (recursive via H2_T22 tunnel 10.0.0.4), 00:02:35
   B       172.31.0.0/25 [200/0] via 172.31.0.1 tag 1 (recursive via H1_T11 tunnel 172.31.1.1), 03:18:41
                 (recursive via H1_T22 tunnel 10.0.0.2), 03:18:41
               [200/0] via 172.31.0.2 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 03:18:41
                 (recursive via H2_T22 tunnel 10.0.0.4), 03:18:41
   ...

   Spoke_1 (root) # get router info routing-table details 10.0.4.0/24
   
   Routing table for VRF=0
   Routing entry for 10.0.4.0/24
     Known via "bgp", distance 200, metric 0, best
     Last update 00:01:11 ago
     * 172.31.0.66, tag 2 (recursive via H2_T11 tunnel 172.31.1.101), tag-match
                 (recursive via H2_T22 tunnel 10.0.0.4), tag-match

5. If traffic is again sent from PC_3 to PC_4, it will go through Hub_2 and reach the destination:

   Spoke_1 (root) # diagnose sniffer packet any 'host 10.0.4.2' 4
   interfaces=[any]
   filters=[host 10.0.4.2]
   7.216948 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
   7.217035 H2_T11 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
   7.217682 H2_T11 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
   7.217729 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply

Example 2: SD-WAN failover when shortcut down

After the shortcut from Spoke_1 to Spoke_2 is established, Spoke_1 will only resolve the PC_4 LAN route to the shortcut, because of best-match resolving, prohibiting SD‑WAN failover. When tag-and-best-match is enabled on Spoke_1, the spoke can resolve the PC_4 LAN route to the shortcut and to other alternative tunnels, allowing SD‑WAN failover.

To test the tag-and-best-match mode:

1. Unset tag-resolve-mode and resume the connections between Spoke_2 and Hub_1. The routing table on Spoke_1 changes to the initial state:

   Spoke_1(root) # get router info routing-table  all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 [2] (recursive via H1_T11 tunnel 172.31.1.1), 00:01:54
                 (recursive via H1_T22 tunnel 10.0.0.2), 00:01:54
                 (recursive via H2_T11 tunnel 172.31.1.101), 00:01:54
                 (recursive via H2_T22 tunnel 10.0.0.4), 00:01:54
   B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 03:30:35
                 (recursive via H1_T22 tunnel 10.0.0.2), 03:30:35
               [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 03:30:35
                 (recursive via H2_T22 tunnel 10.0.0.4), 03:30:35
   S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
               [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
   S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
               [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
   C       172.31.0.65/32 is directly connected, Loopback0
   
   ...

2. Send traffic from PC_3 to PC_4.

   The shortcut from Spoke_1 o Spoke_2 is established.

   The PC_4 LAN route is only resolved to the shortcut because of best-match resolving. If the shortcut is out of SLA, then the traffic cannot switch over to another, alternative tunnel.

   Spoke_1 (root) # get router info routing-table all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 [2] (recursive via H1_T11_0 tunnel 10.0.0.40), 00:09:22
   B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 03:40:12
                 (recursive via H1_T22 tunnel 10.0.0.2), 03:40:12
               [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 03:40:12
                 (recursive via H2_T22 tunnel 10.0.0.4), 03:40:12
   S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
               [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
   S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
               [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
   C       172.31.0.65/32 is directly connected, Loopback0
   S       172.31.0.66/32 [15/0] via H1_T11_0 tunnel 10.0.0.40, [1/0]
   ...

3. If the tag-match mode is set to tag-and-best-match (merge) on Spoke_1, then the PC_4 LAN route is resolved to the H1_T11_0 shortcut based on best-match resolving, and to H1_T11, H1_T22, H2_T11, H2_T22 based on tag-match resolving. It is then resolved to H1_T11, H1_T22, H2_T11, H2_T22 after letting the shortcut hide its parent tunnel.

   Spoke_1 (root) # get router info routing-table  all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 tag 1 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:07:36
                 (recursive via H1_T22 tunnel 10.0.0.2), 00:07:36
               [200/0] via 172.31.0.66 tag 2 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:07:36
                 (recursive via H2_T11 tunnel 172.31.1.101), 00:07:36
                 (recursive via H2_T22 tunnel 10.0.0.4), 00:07:36
   B       172.31.0.0/25 [200/0] via 172.31.0.1 tag 1 (recursive via H1_T11 tunnel 172.31.1.1), 03:48:26
                 (recursive via H1_T22 tunnel 10.0.0.2), 03:48:26
               [200/0] via 172.31.0.2 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 03:48:26
                 (recursive via H2_T22 tunnel 10.0.0.4), 03:48:26
   S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
               [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
   S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
               [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
   C       172.31.0.65/32 is directly connected, Loopback0
   S       172.31.0.66/32 [15/0] via H1_T11_0 tunnel 10.0.0.40, [1/0]
   
   ...
   

   Spoke_1 (root) # get router info routing-table details 10.0.4.0/24
   
   Routing table for VRF=0
   Routing entry for 10.0.4.0/24
     Known via "bgp", distance 200, metric 0, best
     Last update 00:01:02 ago
     * 172.31.0.66, tag 1 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                 (recursive via H1_T22 tunnel 10.0.0.2), tag-match
     * 172.31.0.66, tag 2 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                 (recursive via H2_T11 tunnel 172.31.1.101), tag-match
                 (recursive via H2_T22 tunnel 10.0.0.4), tag-match

4. If the H1_T11_0 shortcut goes out of SLA, traffic will switch to tunnel H1_T22 and shortcut H1_T22_0 is triggered. The PC_4 LAN route is resolved to H1_T11, H1_T22, H2_T11, H2_T22.

   Spoke_1 (root) # get router info routing-table  all
   C       10.0.3.0/24 is directly connected, port4
   B       10.0.4.0/24 [200/0] via 172.31.0.66 tag 1 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:18:50
                 (recursive via H1_T22_0 tunnel 10.0.0.41), 00:18:50
               [200/0] via 172.31.0.66 tag 2 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:18:50
                 (recursive via H1_T22_0 tunnel 10.0.0.41), 00:18:50
                 (recursive via H2_T11 tunnel 172.31.1.101), 00:18:50
                 (recursive via H2_T22 tunnel 10.0.0.4), 00:18:50
   B       172.31.0.0/25 [200/0] via 172.31.0.1 tag 1 (recursive via H1_T11 tunnel 172.31.1.1), 03:59:40
                 (recursive via H1_T22 tunnel 10.0.0.2), 03:59:40
               [200/0] via 172.31.0.2 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 03:59:40
                 (recursive via H2_T22 tunnel 10.0.0.4), 03:59:40
   S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
               [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
   S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
               [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
   C       172.31.0.65/32 is directly connected, Loopback0
   S       172.31.0.66/32 [15/0] via H1_T11_0 tunnel 10.0.0.40, [1/0]
               [15/0] via H1_T22_0 tunnel 10.0.0.41, [1/0]
   ...

   Spoke_1 (root) # get router info routing-table details 10.0.4.0/24
   
   Routing table for VRF=0
   Routing entry for 10.0.4.0/24
     Known via "bgp", distance 200, metric 0, best
     Last update 00:06:40 ago
     * 172.31.0.66, tag 1 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                 (recursive via H1_T22_0 tunnel 10.0.0.43), best-match
     * 172.31.0.66, tag 2 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                 (recursive via H1_T22_0 tunnel 10.0.0.43), best-match
                 (recursive via H2_T11 tunnel 172.31.1.101), tag-match
                 (recursive via H2_T22 tunnel 10.0.0.4), tag-match

   Spoke_1(root) # diagnose sys sdwan service
   
   Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Gen(22), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
     Member sub interface(4):
       1: seq_num(1), interface(H1_T11):
          1: H1_T11_0(93)
       3: seq_num(4), interface(H1_T22):
          1: H1_T22_0(94)
     Members(4):
       1: Seq_num(1 H1_T11), alive, sla(0x1), gid(0), cfg_order(0), cost(0), selected
       2: Seq_num(4 H1_T22_0), alive, sla(0x1), gid(0), cfg_order(3), cost(0), selected
       3: Seq_num(4 H1_T22), alive, sla(0x1), gid(0), cfg_order(3), cost(0), selected
       4: Seq_num(1 H1_T11_0), alive, sla(0x0), gid(0), cfg_order(0), cost(0), selected     
     Src address(1):
           10.0.0.0-10.255.255.255
     Dst address(1):
           10.0.0.0-10.255.255.255
   
   Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
     Gen(10), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
     Members(2):
       1: Seq_num(6 H2_T11), alive, sla(0x1), gid(0), cfg_order(0), cost(0), selected
       2: Seq_num(9 H2_T22), alive, sla(0x1), gid(0), cfg_order(3), cost(0), selected
     Src address(1):
           10.0.0.0-10.255.255.255
     Dst address(1):
           10.0.0.0-10.255.255.255