Dynamic BGP on loopback (Dual-Hub region)

This topic includes reference configurations for the following components:

• Spoke

• Hub

• Optimizing failover

Spoke

1. Configure Loopback to be used for BGP termination and for ADVPN shortcut monitoring:

       config system interface
         edit "Lo"
           set vdom "root"
           set type loopback
           set ip 10.200.1.1/32
           set allowaccess ping
         next
       end

2. Configure a unique Location ID. (Best practice is to use Loopback.)

       config system settings
         set location-id 10.200.1.1
       end

3. Configure IKE to automatically inject the static route to reach the Loopback on all the phase1-interfaces towards the Hub:

       config vpn ipsec phase1-interface
         edit "H1_ISP1"
           set exchange-ip-addr4 10.200.1.1
         next
         edit "H1_MPLS"
           set exchange-ip-addr4 10.200.1.1
         next
         edit "H2_ISP1"
           set exchange-ip-addr4 10.200.1.1
         next
         edit "H2_MPLS"
           set exchange-ip-addr4 10.200.1.1
         next
       end

4. There is no need to configure any tunnel IPs—that is no IPs on the interfaces H1_ISP1, H1_MPLS, H2_ISP1 and H2_MPLS.

5. Configure BGP:

   • Single neighbor per Hub (using the Hub's Loopback) and terminated on the Loopback

   • No ADD-PATH is needed.

   • Must enable set recursive-next-hop enable.

         config router bgp
           set as 65001
           set router-id 10.200.1.1
           set keepalive-timer 15
           set holdtime-timer 45
           set ibgp-multipath enable
           set recursive-next-hop enable
           set graceful-restart enable
           config neighbor
             edit 10.200.1.253
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set interface "Lo"
               set update-source "Lo"
               set connect-timer 1
               set remote-as 65001
             next
             edit 10.200.1.254
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set interface "Lo"
               set update-source "Lo"
               set connect-timer 1
               set remote-as 65001
             next
           end
           config network
             edit 1
               set prefix 10.0.1.0/24
             next
           end
         end

6. Configure dynamic BGP (if desired)

   In a single-region deployment, the following configuration is enough:

   • It enables dynamic IBGP sessions from remote Spokes within the same region.

   • Must enable set passive disable.

         config router bgp
           config neighbor-group
             edit "DYN_EDGE"
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set next-hop-self enable
               set interface "Lo"
               set update-source "Lo"
               set passive disable
               set remote-as 65001
             next    
           end
           config neighbor-range
             edit 101
               set prefix 10.200.0.0/14
               set neighbor-group "DYN_EDGE"
             next
           end  
         end

   To make the configuration ready for a multi-regional deployment, additional steps are necessary:

   • Must use set remote-as-filter to enable both IBGP and EBGP sessions from remote Spokes.

   • Must tag the local LAN prefixes to restrict route advertisement via dynamic EBGP sessions.

         config router aspath-list
             edit "SDWAN_AS"
               config rule
                 edit 1
                   set action permit
                   set regexp "6500."
                 next
               end
             next
           end

         config router route-map
             edit "LAN_TAG"
               config rule
                 edit 1
                   set set-tag 100
                 next
               end
             next
             edit "LAN_OUT"
               config rule
                 edit 1
                   set match-tag 100
                 next
               end  
             next
           end

         config router bgp
             config neighbor-group
               edit "DYN_EDGE"
                 set ebgp-enforce-multihop enable 
                 set soft-reconfiguration enable
                 set capability-graceful-restart enable
                 set advertisement-interval 1
                 set next-hop-self enable
                 set interface "Lo"
                 set update-source "Lo"
                 set passive disable
                 set remote-as-filter "SDWAN_AS"
                 set route-map-out "LAN_OUT"
               next    
             end
             config neighbor-range
               edit 101
                 set prefix 10.200.0.0/14
                 set neighbor-group "DYN_EDGE"
               next
             end
             config network
               edit 1
                 set prefix 10.0.1.0/24
                 set route-map "LAN_TAG"
               next
             end
           end

7. In SD-WAN Member configuration, must set set source on all the overlays, to ensure that the Loopback IP is used as a source of health probes:

       config system sdwan
         config members
           edit 2
             set interface "H1_ISP1"
             set source 10.200.1.1
           next
           edit 3
             set interface "H1_MPLS"
             set source 10.200.1.1
           next
           edit 4
             set interface "H2_ISP1"        
             set source 10.200.1.1
           next
           edit 5
             set interface "H2_MPLS"
             set source 10.200.1.1
           next
         end
       end

8. In Firewall Policy configuration, must add a rule to permit incoming health probes (for the ADVPN shortcut monitoring) and Dynamic BGP sessions (if enabled), both destined to the loopback:

       config firewall policy
         edit 4
           set name "Peering"
           set srcintf "overlay"
           set dstintf "Lo"
           set srcaddr "all"
           set dstaddr "all"
           set action accept
           set schedule "always"
           set service "PING" "BGP"
         next
       end

Hub

1. Configure Loopback to be used for BGP termination:

       config system interface
         edit "Lo"
           set vdom "root"
           set type loopback
           set ip 10.200.1.253/32
           set allowaccess ping
         next
       end

   DO NOT use the same loopback as Spokes' health-check server! Using the same loopback is not supported, because the healh-check routes are statically injected on the Spokes, and they remain even when the respective overlay is down. As a result, the BGP session will be unable to switchover to another overlay upon failure!

2. Configure a unique Location ID. Best practice is to use Loopback.

       config system settings
         set location-id 10.200.1.253
       end

3. Configure IKE to automatically inject the static route to reach the Loopback on all the Dial-Up phase1-interfaces towards the Spokes:

       config vpn ipsec phase1-interface
         edit "EDGE_ISP1"
           set exchange-ip-addr4 10.200.1.253
         next
         edit "EDGE_MPLS"
           set exchange-ip-addr4 10.200.1.253
         next
       end

4. There is no need to configure any tunnel IPs—that is, no IPs on the interfaces EDGE_ISP1 and EDGE_MPLS.

5. Configure BGP:

   • Single neighbor-group for all Spokes and terminated on the Loopback.

   • No ADD-PATH is needed.

   • No RR is needed, if Dynamic BGP is enabled on the Spokes.

   • Must configure set recursive-next-hop enable.

   • Must advertise a loopback summary to all the Spokes (10.200.0.0/14 in the snippet below), so that they can resolve each other's routes.

         config router route-map
           edit "LOCAL_REGION"
             config rule
               edit 1
                 set set-community "no-export"
               next
             end
           next
         end    
         config router bgp
           set as 65001
           set router-id 10.200.1.253
           set keepalive-timer 15
           set holdtime-timer 45
           set ibgp-multipath enable
           set ebgp-multipath enable
           set recursive-next-hop enable
           set graceful-restart enable
           config neighbor-group
             edit "EDGE"
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set next-hop-self enable
               set remote-as 65001
               set interface "Lo"
               set update-source "Lo"
             next
           end
           config neighbor-range
             edit 1
               set prefix 10.200.0.0/14
               set neighbor-group "EDGE"
             next
           end
           config network
             edit 1
               set prefix 10.200.0.0/14
               set route-map "LOCAL_REGION"
             next
             edit 2
               set prefix 10.1.0.0/24
             next
           end
         end

   If Dynamic BGP is NOT used, then RR must be enabled on the neighbor-group for the correct ADVPN operation: config router bgp config neighbor-group edit "EDGE" set route-reflector-client enable next end end

6. In Firewall Policy configuration, must add a rule to permit incoming BGP sessions destined to the Loopback:

       config firewall policy
         edit 6
           set name "Peering"
           set srcintf overlay
           set dstintf "Lo"
           set srcaddr "all"
           set dstaddr "all"
           set action accept
           set schedule "always"
           set service "PING" "BGP"
         next
       end

Optimizing failover

The following best-practice recommendations improve failover times during different network issues:

• On all devices, DPD timers must be shorter than BGP hold-timer. This way, if one overlay fails, the BGP session will seamlessly switchover to another available overlay.

• On the Hub, dpd on-idle is recommended, with timers shorter than those on the Spokes. This will allow the Hub to quickly detect a failed overlay on a Spoke, to avoid resolving routes through dead overlays.

• It is recommended to configure default route on the Spokes towards the overlays. One easy way of doing this is using set sdwan-zone "underlay" "overlay" static route on the Spokes.

Please note that the deployment workflow discussed throughout this document already incorporates all the above recommendations.