BGP next hop tag-match mode

Tag-match mode can be configured to increase flexibility when controlling how BGP routes' next hops are resolved:

config router bgp
    set tag-resolve-mode {disable | preferred | merge}
end

Best-match (disable)

Resolve the BGP route's next hops with best-matched routes. This is the default setting.

Tag-match (preferred)

Resolve the BGP route's next hops with routes that have the same tag. If there are no results, resolve the next hops with best-matched routes.

Tag-and-best-match (merge)

Merge tag-match with best-match if they are using different routes, then let shortcuts hide their parents. The results exclude the next hops of tag-match whose interfaces have appeared in best-match.

In these examples:

  • Each spoke has two IPsec tunnels to each hub, and one BGP peer on loopback interface to each hub (route-reflector).

  • The loopbacks are exchanged with IKE between the spokes and hubs. They are installed as static routes that are used to provide reachability for establishing BGP neighbors.

  • The summary BGP routes from the loopback IP address ranges that originated on the hubs are advertised to the spokes for resolving the BGP next hop s on the spokes.

  • The spokes' PC LAN subnets are reflected by the hubs.

  • Spoke_1 receives BGP routes (the LAN subnet and loopback IP summary) from Hub_1 with tag 1 and from Hub_2 with tag 2.

  • SD-WAN is enabled on Spoke_1, and all of the tunnels are SD-WAN members.

Example 1: Connection between Hub and Spoke down

If the connections between Hub_1 and Spoke_2 are down, traffic from PC_3 to PC_4 can still go through Hub_1 because of the best-match resolving on Spoke_1, but packets will be dropped on Hub_1. When tag-match is enabled on Spoke_1, the spoke will resolve the PC_4 LAN route to Hub2, and traffic will be forwarded to Hub_2 and reach its destination.

To test the tag-match mode:
  1. View the key routes on Spoke_1:

    Spoke_1(root) # get router info routing-table all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 [2] (recursive via H1_T11 tunnel 172.31.1.1), 20:09:52
                  (recursive via H1_T22 tunnel 10.0.0.2), 20:09:52
                  (recursive via H2_T11 tunnel 172.31.1.101), 20:09:52
                  (recursive via H2_T22 tunnel 10.0.0.4), 20:09:52
    B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 23:25:37
                  (recursive via H1_T22 tunnel 10.0.0.2), 23:25:37
                [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 23:25:37
                  (recursive via H2_T22 tunnel 10.0.0.4), 23:25:37
    S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
                [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
    S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
                [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
    C       172.31.0.65/32 is directly connected, Loopback0
    ...

    172.31.0.0/25 is the loopback IP summary originated by both Hub_1 and Hub_2. The next hop of the PC_4 LAN route is resolved to Hub_1 (H1_T11, H1_T22) and Hub_2 (H2_T11, H2_T22) based on the loopback IP summary route.

  2. When connections between Spoke_2 and Hub_1 fails due to the BGP neighbor, tunnels, or physical ports going down, the PC_4 LAN route can be still resolved to Hub_1 and Hub_2 because the loopback IP summary can still be received from both Hub_1 and Hub_2:

    Spoke_1(root) # get router info routing-table all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 (recursive via H1_T11 tunnel 172.31.1.1), 00:03:06
                  (recursive via H1_T22 tunnel 10.0.0.2), 00:03:06
                  (recursive via H2_T11 tunnel 172.31.1.101), 00:03:06
                  (recursive via H2_T22 tunnel 10.0.0.4), 00:03:06
    B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 23:55:34
                  (recursive via H1_T22 tunnel 10.0.0.2), 23:55:34
                [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 23:55:34
                  (recursive via H2_T22 tunnel 10.0.0.4), 23:55:34
    ...
  3. If traffic sent from PC_3 to PC_4 goes through Hub_1, packets are dropped because there is no PC_4 LAN route on Hub_1:

    Spoke_1 (root) # diagnose sniffer packet any 'host 10.0.4.2' 4
    interfaces=[any]
    filters=[host 10.0.4.2]
    11.261264 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
    11.261349 H1_T11 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
    12.260268 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
    12.260291 H1_T11 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
    
    Hub_1 (root) #  diagnose sniffer packet any 'host 10.0.4.2' 4
    interfaces=[any]
    filters=[host 10.0.4.2]
    6.966064 EDGE_T1 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
    7.965012 EDGE_T1 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
  4. If the tag-match mode is set to tag-match (preferred) on Spoke_1, then the PC_4 LAN route can only be resolved to Hub_2 because of tag-match checking:

    Spoke_1(root) # get router info routing-table all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 00:02:35
                  (recursive via H2_T22 tunnel 10.0.0.4), 00:02:35
    B       172.31.0.0/25 [200/0] via 172.31.0.1 tag 1 (recursive via H1_T11 tunnel 172.31.1.1), 03:18:41
                  (recursive via H1_T22 tunnel 10.0.0.2), 03:18:41
                [200/0] via 172.31.0.2 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 03:18:41
                  (recursive via H2_T22 tunnel 10.0.0.4), 03:18:41
    ...
    Spoke_1 (root) # get router info routing-table details 10.0.4.0/24
    
    Routing table for VRF=0
    Routing entry for 10.0.4.0/24
      Known via "bgp", distance 200, metric 0, best
      Last update 00:01:11 ago
      * 172.31.0.66, tag 2 (recursive via H2_T11 tunnel 172.31.1.101), tag-match
                  (recursive via H2_T22 tunnel 10.0.0.4), tag-match
  5. If traffic is again sent from PC_3 to PC_4, it will go through Hub_2 and reach the destination:

    Spoke_1 (root) # diagnose sniffer packet any 'host 10.0.4.2' 4
    interfaces=[any]
    filters=[host 10.0.4.2]
    7.216948 port4 in 10.0.3.2 -> 10.0.4.2: icmp: echo request
    7.217035 H2_T11 out 10.0.3.2 -> 10.0.4.2: icmp: echo request
    7.217682 H2_T11 in 10.0.4.2 -> 10.0.3.2: icmp: echo reply
    7.217729 port4 out 10.0.4.2 -> 10.0.3.2: icmp: echo reply

Example 2: SD-WAN failover when shortcut down

After the shortcut from Spoke_1 to Spoke_2 is established, Spoke_1 will only resolve the PC_4 LAN route to the shortcut, because of best-match resolving, prohibiting SD‑WAN failover. When tag-and-best-match is enabled on Spoke_1, the spoke can resolve the PC_4 LAN route to the shortcut and to other alternative tunnels, allowing SD‑WAN failover.

To test the tag-and-best-match mode:
  1. Unset tag-resolve-mode and resume the connections between Spoke_2 and Hub_1. The routing table on Spoke_1 changes to the initial state:

    Spoke_1(root) # get router info routing-table  all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 [2] (recursive via H1_T11 tunnel 172.31.1.1), 00:01:54
                  (recursive via H1_T22 tunnel 10.0.0.2), 00:01:54
                  (recursive via H2_T11 tunnel 172.31.1.101), 00:01:54
                  (recursive via H2_T22 tunnel 10.0.0.4), 00:01:54
    B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 03:30:35
                  (recursive via H1_T22 tunnel 10.0.0.2), 03:30:35
                [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 03:30:35
                  (recursive via H2_T22 tunnel 10.0.0.4), 03:30:35
    S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
                [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
    S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
                [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
    C       172.31.0.65/32 is directly connected, Loopback0
    
    ...
  2. Send traffic from PC_3 to PC_4.

    The shortcut from Spoke_1 o Spoke_2 is established.

    The PC_4 LAN route is only resolved to the shortcut because of best-match resolving. If the shortcut is out of SLA, then the traffic cannot switch over to another, alternative tunnel.

    Spoke_1 (root) # get router info routing-table all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 [2] (recursive via H1_T11_0 tunnel 10.0.0.40), 00:09:22
    B       172.31.0.0/25 [200/0] via 172.31.0.1 (recursive via H1_T11 tunnel 172.31.1.1), 03:40:12
                  (recursive via H1_T22 tunnel 10.0.0.2), 03:40:12
                [200/0] via 172.31.0.2 (recursive via H2_T11 tunnel 172.31.1.101), 03:40:12
                  (recursive via H2_T22 tunnel 10.0.0.4), 03:40:12
    S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
                [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
    S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
                [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
    C       172.31.0.65/32 is directly connected, Loopback0
    S       172.31.0.66/32 [15/0] via H1_T11_0 tunnel 10.0.0.40, [1/0]
    ...
  3. If the tag-match mode is set to tag-and-best-match (merge) on Spoke_1, then the PC_4 LAN route is resolved to the H1_T11_0 shortcut based on best-match resolving, and to H1_T11, H1_T22, H2_T11, H2_T22 based on tag-match resolving. It is then resolved to H1_T11, H1_T22, H2_T11, H2_T22 after letting the shortcut hide its parent tunnel.

    Spoke_1 (root) # get router info routing-table  all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 tag 1 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:07:36
                  (recursive via H1_T22 tunnel 10.0.0.2), 00:07:36
                [200/0] via 172.31.0.66 tag 2 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:07:36
                  (recursive via H2_T11 tunnel 172.31.1.101), 00:07:36
                  (recursive via H2_T22 tunnel 10.0.0.4), 00:07:36
    B       172.31.0.0/25 [200/0] via 172.31.0.1 tag 1 (recursive via H1_T11 tunnel 172.31.1.1), 03:48:26
                  (recursive via H1_T22 tunnel 10.0.0.2), 03:48:26
                [200/0] via 172.31.0.2 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 03:48:26
                  (recursive via H2_T22 tunnel 10.0.0.4), 03:48:26
    S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
                [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
    S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
                [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
    C       172.31.0.65/32 is directly connected, Loopback0
    S       172.31.0.66/32 [15/0] via H1_T11_0 tunnel 10.0.0.40, [1/0]
    
    ...
    
    Spoke_1 (root) # get router info routing-table details 10.0.4.0/24
    
    Routing table for VRF=0
    Routing entry for 10.0.4.0/24
      Known via "bgp", distance 200, metric 0, best
      Last update 00:01:02 ago
      * 172.31.0.66, tag 1 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                  (recursive via H1_T22 tunnel 10.0.0.2), tag-match
      * 172.31.0.66, tag 2 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                  (recursive via H2_T11 tunnel 172.31.1.101), tag-match
                  (recursive via H2_T22 tunnel 10.0.0.4), tag-match
  4. If the H1_T11_0 shortcut goes out of SLA, traffic will switch to tunnel H1_T22 and shortcut H1_T22_0 is triggered. The PC_4 LAN route is resolved to H1_T11, H1_T22, H2_T11, H2_T22.

    Spoke_1 (root) # get router info routing-table  all
    C       10.0.3.0/24 is directly connected, port4
    B       10.0.4.0/24 [200/0] via 172.31.0.66 tag 1 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:18:50
                  (recursive via H1_T22_0 tunnel 10.0.0.41), 00:18:50
                [200/0] via 172.31.0.66 tag 2 (recursive via H1_T11_0 tunnel 10.0.0.40), 00:18:50
                  (recursive via H1_T22_0 tunnel 10.0.0.41), 00:18:50
                  (recursive via H2_T11 tunnel 172.31.1.101), 00:18:50
                  (recursive via H2_T22 tunnel 10.0.0.4), 00:18:50
    B       172.31.0.0/25 [200/0] via 172.31.0.1 tag 1 (recursive via H1_T11 tunnel 172.31.1.1), 03:59:40
                  (recursive via H1_T22 tunnel 10.0.0.2), 03:59:40
                [200/0] via 172.31.0.2 tag 2 (recursive via H2_T11 tunnel 172.31.1.101), 03:59:40
                  (recursive via H2_T22 tunnel 10.0.0.4), 03:59:40
    S       172.31.0.1/32 [15/0] via H1_T11 tunnel 172.31.1.1, [1/0]
                [15/0] via H1_T22 tunnel 10.0.0.2, [1/0]
    S       172.31.0.2/32 [15/0] via H2_T11 tunnel 172.31.1.101, [1/0]
                [15/0] via H2_T22 tunnel 10.0.0.4, [1/0]
    C       172.31.0.65/32 is directly connected, Loopback0
    S       172.31.0.66/32 [15/0] via H1_T11_0 tunnel 10.0.0.40, [1/0]
                [15/0] via H1_T22_0 tunnel 10.0.0.41, [1/0]
    ...
    Spoke_1 (root) # get router info routing-table details 10.0.4.0/24
    
    Routing table for VRF=0
    Routing entry for 10.0.4.0/24
      Known via "bgp", distance 200, metric 0, best
      Last update 00:06:40 ago
      * 172.31.0.66, tag 1 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                  (recursive via H1_T22_0 tunnel 10.0.0.43), best-match
      * 172.31.0.66, tag 2 (recursive via H1_T11_0 tunnel 10.0.0.42), best-match
                  (recursive via H1_T22_0 tunnel 10.0.0.43), best-match
                  (recursive via H2_T11 tunnel 172.31.1.101), tag-match
                  (recursive via H2_T22 tunnel 10.0.0.4), tag-match
    Spoke_1(root) # diagnose sys sdwan service
    
    Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
      Gen(22), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Member sub interface(4):
        1: seq_num(1), interface(H1_T11):
           1: H1_T11_0(93)
        3: seq_num(4), interface(H1_T22):
           1: H1_T22_0(94)
      Members(4):
        1: Seq_num(1 H1_T11), alive, sla(0x1), gid(0), cfg_order(0), cost(0), selected
        2: Seq_num(4 H1_T22_0), alive, sla(0x1), gid(0), cfg_order(3), cost(0), selected
        3: Seq_num(4 H1_T22), alive, sla(0x1), gid(0), cfg_order(3), cost(0), selected
        4: Seq_num(1 H1_T11_0), alive, sla(0x0), gid(0), cfg_order(0), cost(0), selected     
      Src address(1):
            10.0.0.0-10.255.255.255
      Dst address(1):
            10.0.0.0-10.255.255.255
    
    Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
      Gen(10), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
      Members(2):
        1: Seq_num(6 H2_T11), alive, sla(0x1), gid(0), cfg_order(0), cost(0), selected
        2: Seq_num(9 H2_T22), alive, sla(0x1), gid(0), cfg_order(3), cost(0), selected
      Src address(1):
            10.0.0.0-10.255.255.255
      Dst address(1):
            10.0.0.0-10.255.255.255