Controlling traffic with BGP route mapping and service rules

SD-WAN allows you to select different outbound WAN links based on performance SLAs. It is important that BGP neighbors are aware of these settings, and changes to them.

BGP can adapt to changes in SD-WAN link SLAs in the following ways:

• Applying different route-maps based on the SD-WAN's health checks. For example, different BGP community strings can be advertised to BGP neighbors when SLAs are not met.
• Traffic can be selectively forwarded based on the active BGP neighbor. If the SD-WAN service's role matches the active SD-WAN neighbor, the service is enabled. If there is no match, then the service is disabled.

Example

In this topology, a branch FortiGate has two SD-WAN gateways serving as the primary and secondary gateways. The gateways reside in different datacenters, but have a full mesh network between them.

This example shows how route-maps and service rules are selected based on performance SLAs and the member that is currently active. Traffic flows through the primary gateway unless the neighbor's health check is outside of its SLA. If that happens, traffic routes to the secondary gateway.

BGP NBR1 is the primary neighbor and BGP NBR2 is the secondary neighbor.

The branch FortiGate's wan1 and wan2 interfaces are members of the SD-WAN. When the SD-WAN neighbor status is primary, it will advertise community 20:1 to BGP NBR1 and 20:5 to BGP NBR2. When the SD-WAN neighbor status is secondary, it will advertise 20:5 to BGP NBR1 and 20:2 to BGP NBR2.

Only one of the primary or secondary neighbors can be active at one time. The SD-WAN neighbor status is used to decide which neighbor is selected:

• Primary: The primary neighbor takes precedence if its SLAs are met.
• Secondary: If the primary neighbor's SLAs are not met, the secondary neighbor becomes active if its SLAs are met.
• Standalone: If neither the primary or secondary neighbor's SLAs are met, the SD-WAN neighbor status becomes standalone.

Route map

SD-WAN is configured to let BGP advertise different communities when the SLA status changes. When the SLA is missed, it triggers BGP to advertise a different community to its BGP neighbor based on its route-map. The BGP neighbors can use the received community string to select the best path to reach the branch.

To configure BGP route-maps and neighbors:

1. Configure an access for the routes to be matched:

   config router access-list
       edit "net192"
           config rule
               edit 1
                   set prefix 192.168.20.0 255.255.255.0
               next
           end
       next
   end

2. Configure the primary neighbor's preferred route-map:

   config router route-map
       edit "comm1"
           config rule
               edit 1
                   set match-ip-address "net192"
                   set set-community "20:1"
               next
           end
       next
   end
   

3. Configure the secondary neighbor's preferred route-map:

   config router route-map
       edit "comm2"
           config rule
               edit 1
                   set match-ip-address "net192"
                   set set-community "20:2"
               next
           end
       next
   end
   

4. Configure the failed route-map:

   config router route-map
       edit "comm5"
           config rule
               edit 1
                   set match-ip-address "net192"
                   set set-community "20:5"
               next
           end
       next
   end
   

5. Configure BGP neighbors:

   config router bgp
       set as 65412
       set router-id 1.1.1.1
       set ibgp-multipath enable
       config neighbor
           edit "10.100.1.1"
               set soft-reconfiguration enable
               set remote-as 20
               set route-map-out "comm5"
               set route-map-out-preferable "comm1"
           next
           edit "10.100.1.5"
               set soft-reconfiguration enable
               set remote-as 20
               set route-map-out "comm5"
               set route-map-out-preferable "comm2"
           next
       end
   end

   When SLAs are met, route-map-out-preferable is used. When SLAs are missed, route-map-out is used.

To configure SD-WAN:

1. Configure the SD-WAN members:

   config system sdwan
       set status enable
       config members
           edit 1
               set interface "port1"
           next
           edit 2
               set interface "port2"
           next
       end
   end

2. Configure health checks for each member:

   config system sdwan
       config health-check
           edit "ping"
               set server "10.100.2.22"
               set members 1
               config sla
                   edit 1
                       set link-cost-factor packet-loss
                       set packetloss-threshold 1
                   next
               end
           next
           edit "ping2"
               set server "10.100.2.23"
               set members 2
               config sla
                   edit 1
                       set link-cost-factor packet-loss
                       set packetloss-threshold 1
                   next
               end
           next
       end
   end

3. Configure the SD-WAN neighbors and assign them a role and the health checks used to determine if the neighbor meets the SLA:

   SD-WAN neighbors can only be configured in the CLI.

   config system sdwan
       config neighbor
           edit "10.100.1.1"
               set member 1
               set role primary
               set health-check "ping"
               set sla-id 1
           next
           edit "10.100.1.5"
               set member 2
               set role secondary
               set health-check "ping2"
               set sla-id 1
           next
       end
   end

Service rules

Create SD-WAN service rules to direct traffic to the primary neighbor when its SLAs are met, and to the secondary neighbor when the primary neighbor's SLAs are missed.

To configure the SD-WAN service rules:

config system sdwan
    config service
        edit 1
            set name "Primary-Out"
            set role primary
            set dst "all"
            set src "all"
            set priority-members 1
        next
        edit 2
            set name "Secondary-Out"
            set role secondary
            set dst "all"
            set src "all"
            set priority-members 2
        next
    end
end

If neither the primary nor secondary neighbors are active, the SD-WAN neighbor status becomes standalone. Only service rules with standalone-action enabled will continue to pass traffic. This option is disabled by default.

Verification

To verify when the primary neighbor is passing traffic:

1. Verify the health check status:

   FortiGate-Branch # diagnose sys sdwan health-check
   Health Check(ping):
   Seq(1 port1): state(alive), packet-loss(0.000%) latency(0.569), jitter(0.061) sla_map=0x1
   Health Check(ping2):
   Seq(2 port2): state(alive), packet-loss(0.000%) latency(3.916), jitter(2.373) sla_map=0x1
   

2. Verify SD-WAN neighbor status:

   FortiGate-Branch # diagnose sys sdwan neighbor
   SD-WAN neighbor status: hold-down(disable), hold-down-time(0), hold_boot_time(0)
           Selected role(primary) last_secondary_select_time/current_time in seconds 0/572
   Neighbor(10.100.1.1): member(1) role(primary)
           Health-check(ping:1)  sla-pass selected alive
   Neighbor(10.100.1.5): member(2) role(secondary)
           Health-check(ping2:1)  sla-pass alive

3. Verify service rules status:

   FortiGate-Branch # diagnose sys sdwan service
   
   Service(1): Address Mode(IPV4) flags=0x0
     Gen(3), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
     Service role: primary
     Members:
       1: Seq_num(1 port1), alive, selected
     Src address:
           0.0.0.0-255.255.255.255
   
     Dst address:
           0.0.0.0-255.255.255.255
   
   Service(2): Address Mode(IPV4) flags=0x0
     Gen(6), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
     Service role: secondary, disabled by unselected.
     Members:
       1: Seq_num(2 port2), alive, selected
     Src address:
           0.0.0.0-255.255.255.255
   
     Dst address:
           0.0.0.0-255.255.255.255

4. Verify neighbor routers:
   1. Primary neighbor router:

      FGT-NBR1 # get router info bgp network 192.168.20.0
      BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        64512
          10.100.1.2 from 10.100.1.2 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:1
            Last update: Thu Apr 30 13:41:40 2020

   2. Secondary neighbor router:

      FGT-NBR2 # get router info bgp network 192.168.20.0
      VRF 0 BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        Original VRF 0
        64512
          10.100.1.6 from 10.100.1.6 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:5
            Last update: Thu Apr 30 13:41:39 2020
      

To verify when the secondary neighbor is passing traffic:

1. Verify the health check status:

   FortiGate-Branch # diagnose sys sdwan health-check
   Health Check(ping):
   Seq(1 port1): state(dead), packet-loss(54.000%) sla_map=0x0
   Health Check(ping2):
   Seq(2 port2): state(alive), packet-loss(0.000%) latency(4.339), jitter(3.701) sla_map=0x1

2. Verify SD-WAN neighbor status:

   FortiGate-Branch # diagnose sys sdwan neighbor
   SD-WAN neighbor status: hold-down(disable), hold-down-time(0), hold_boot_time(0)
           Selected role(secondary) last_secondary_select_time/current_time in seconds 936/936
   Neighbor(10.100.1.1): member(1) role(primary)
           Health-check(ping:1)  sla-fail dead
   Neighbor(10.100.1.5): member(2) role(secondary)
           Health-check(ping2:1)  sla-pass selected alive

3. Verify service rules status:

   FortiGate-Branch # diagnose sys sdwan service
   
   Service(1): Address Mode(IPV4) flags=0x0
     Gen(4), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
     Service role: primary, disabled by unselected.
     Members:
       1: Seq_num(1 port1), alive, selected
     Src address:
           0.0.0.0-255.255.255.255
   
     Dst address:
           0.0.0.0-255.255.255.255
   
   Service(2): Address Mode(IPV4) flags=0x0
     Gen(7), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
     Service role: secondary
     Members:
       1: Seq_num(2 port2), alive, selected
     Src address:
           0.0.0.0-255.255.255.255
   
     Dst address:
           0.0.0.0-255.255.255.255
   

4. Verify neighbor routers:
   1. Primary neighbor router:

      FGT-NBR1 # get router info bgp network 192.168.20.0
      BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        64512
          10.100.1.2 from 10.100.1.2 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:5
            Last update: Thu Apr 30 15:41:58 2020
      

   2. Secondary neighbor router:

      FGT-NBR2 # get router info bgp network 192.168.20.0
      VRF 0 BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        Original VRF 0
        64512
          10.100.1.6 from 10.100.1.6 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:2
            Last update: Thu Apr 30 15:42:07 2020