Fortinet black logo

Administration Guide

Controlling traffic with BGP route mapping and service rules

Controlling traffic with BGP route mapping and service rules

SD-WAN allows you to select different outbound WAN links based on performance SLAs. It is important that BGP neighbors are aware of these settings, and changes to them.

BGP can adapt to changes in SD-WAN link SLAs in the following ways:

  • Applying different route-maps based on the SD-WAN's health checks. For example, different BGP community strings can be advertised to BGP neighbors when SLAs are not met.
  • Traffic can be selectively forwarded based on the active BGP neighbor. If the SD-WAN service's role matches the active SD-WAN neighbor, the service is enabled. If there is no match, then the service is disabled.

Example

In this topology, a branch FortiGate has two SD-WAN gateways serving as the primary and secondary gateways. The gateways reside in different datacenters, but have a full mesh network between them.

This example shows how route-maps and service rules are selected based on performance SLAs and the member that is currently active. Traffic flows through the primary gateway unless the neighbor's health check is outside of its SLA. If that happens, traffic routes to the secondary gateway.

BGP NBR1 is the primary neighbor and BGP NBR2 is the secondary neighbor.

The branch FortiGate's wan1 and wan2 interfaces are members of the SD-WAN. When the SD-WAN neighbor status is primary, it will advertise community 20:1 to BGP NBR1 and 20:5 to BGP NBR2. When the SD-WAN neighbor status is secondary, it will advertise 20:5 to BGP NBR1 and 20:2 to BGP NBR2.

Only one of the primary or secondary neighbors can be active at one time. The SD-WAN neighbor status is used to decide which neighbor is selected:

  • Primary: The primary neighbor takes precedence if its SLAs are met.
  • Secondary: If the primary neighbor's SLAs are not met, the secondary neighbor becomes active if its SLAs are met.
  • Standalone: If neither the primary or secondary neighbor's SLAs are met, the SD-WAN neighbor status becomes standalone.

Route map

SD-WAN is configured to let BGP advertise different communities when the SLA status changes. When the SLA is missed, it triggers BGP to advertise a different community to its BGP neighbor based on its route-map. The BGP neighbors can use the received community string to select the best path to reach the branch.

To configure BGP route-maps and neighbors:
  1. Configure an access for the routes to be matched:
    config router access-list
        edit "net192"
            config rule
                edit 1
                    set prefix 192.168.20.0 255.255.255.0
                next
            end
        next
    end
  2. Configure the primary neighbor's preferred route-map:
    config router route-map
        edit "comm1"
            config rule
                edit 1
                    set match-ip-address "net192"
                    set set-community "20:1"
                next
            end
        next
    end
    
  3. Configure the secondary neighbor's preferred route-map:
    config router route-map
        edit "comm2"
            config rule
                edit 1
                    set match-ip-address "net192"
                    set set-community "20:2"
                next
            end
        next
    end
    
  4. Configure the failed route-map:
    config router route-map
        edit "comm5"
            config rule
                edit 1
                    set match-ip-address "net192"
                    set set-community "20:5"
                next
            end
        next
    end
    
  5. Configure BGP neighbors:
    config router bgp
        set as 65412
        set router-id 1.1.1.1
        set ibgp-multipath enable
        config neighbor
            edit "10.100.1.1"
                set soft-reconfiguration enable
                set remote-as 20
                set route-map-out "comm5"
                set route-map-out-preferable "comm1"
            next
            edit "10.100.1.5"
                set soft-reconfiguration enable
                set remote-as 20
                set route-map-out "comm5"
                set route-map-out-preferable "comm2"
            next
        end
    end

    When SLAs are met, route-map-out-preferable is used. When SLAs are missed, route-map-out is used.

To configure SD-WAN:
  1. Configure the SD-WAN members:
    config system sdwan
        set status enable
        config members
            edit 1
                set interface "port1"
            next
            edit 2
                set interface "port2"
            next
        end
    end
  2. Configure health checks for each member:
    config system sdwan
        config health-check
            edit "ping"
                set server "10.100.2.22"
                set members 1
                config sla
                    edit 1
                        set link-cost-factor packet-loss
                        set packetloss-threshold 1
                    next
                end
            next
            edit "ping2"
                set server "10.100.2.23"
                set members 2
                config sla
                    edit 1
                        set link-cost-factor packet-loss
                        set packetloss-threshold 1
                    next
                end
            next
        end
    end
  3. Configure the SD-WAN neighbors and assign them a role and the health checks used to determine if the neighbor meets the SLA:

    SD-WAN neighbors can only be configured in the CLI.

    config system sdwan
        config neighbor
            edit "10.100.1.1"
                set member 1
                set role primary
                set health-check "ping"
                set sla-id 1
            next
            edit "10.100.1.5"
                set member 2
                set role secondary
                set health-check "ping2"
                set sla-id 1
            next
        end
    end

Service rules

Create SD-WAN service rules to direct traffic to the primary neighbor when its SLAs are met, and to the secondary neighbor when the primary neighbor's SLAs are missed.

To configure the SD-WAN service rules:
config system sdwan
    config service
        edit 1
            set name "Primary-Out"
            set role primary
            set dst "all"
            set src "all"
            set priority-members 1
        next
        edit 2
            set name "Secondary-Out"
            set role secondary
            set dst "all"
            set src "all"
            set priority-members 2
        next
    end
end
Tooltip

If neither the primary nor secondary neighbors are active, the SD-WAN neighbor status becomes standalone. Only service rules with standalone-action enabled will continue to pass traffic. This option is disabled by default.

Verification

To verify when the primary neighbor is passing traffic:
  1. Verify the health check status:
    FortiGate-Branch # diagnose sys sdwan health-check
    Health Check(ping):
    Seq(1 port1): state(alive), packet-loss(0.000%) latency(0.569), jitter(0.061) sla_map=0x1
    Health Check(ping2):
    Seq(2 port2): state(alive), packet-loss(0.000%) latency(3.916), jitter(2.373) sla_map=0x1
    
  2. Verify SD-WAN neighbor status:
    FortiGate-Branch # diagnose sys sdwan neighbor
    SD-WAN neighbor status: hold-down(disable), hold-down-time(0), hold_boot_time(0)
            Selected role(primary) last_secondary_select_time/current_time in seconds 0/572
    Neighbor(10.100.1.1): member(1) role(primary)
            Health-check(ping:1)  sla-pass selected alive
    Neighbor(10.100.1.5): member(2) role(secondary)
            Health-check(ping2:1)  sla-pass alive
  3. Verify service rules status:
    FortiGate-Branch # diagnose sys sdwan service
    
    Service(1): Address Mode(IPV4) flags=0x0
      Gen(3), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: primary
      Members:
        1: Seq_num(1 port1), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
    
    Service(2): Address Mode(IPV4) flags=0x0
      Gen(6), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: secondary, disabled by unselected.
      Members:
        1: Seq_num(2 port2), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
  4. Verify neighbor routers:
    1. Primary neighbor router:
      FGT-NBR1 # get router info bgp network 192.168.20.0
      BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        64512
          10.100.1.2 from 10.100.1.2 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:1
            Last update: Thu Apr 30 13:41:40 2020
    2. Secondary neighbor router:
      FGT-NBR2 # get router info bgp network 192.168.20.0
      VRF 0 BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        Original VRF 0
        64512
          10.100.1.6 from 10.100.1.6 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:5
            Last update: Thu Apr 30 13:41:39 2020
      
To verify when the secondary neighbor is passing traffic:
  1. Verify the health check status:
    FortiGate-Branch # diagnose sys sdwan health-check
    Health Check(ping):
    Seq(1 port1): state(dead), packet-loss(54.000%) sla_map=0x0
    Health Check(ping2):
    Seq(2 port2): state(alive), packet-loss(0.000%) latency(4.339), jitter(3.701) sla_map=0x1
  2. Verify SD-WAN neighbor status:
    FortiGate-Branch # diagnose sys sdwan neighbor
    SD-WAN neighbor status: hold-down(disable), hold-down-time(0), hold_boot_time(0)
            Selected role(secondary) last_secondary_select_time/current_time in seconds 936/936
    Neighbor(10.100.1.1): member(1) role(primary)
            Health-check(ping:1)  sla-fail dead
    Neighbor(10.100.1.5): member(2) role(secondary)
            Health-check(ping2:1)  sla-pass selected alive
  3. Verify service rules status:
    FortiGate-Branch # diagnose sys sdwan service
    
    Service(1): Address Mode(IPV4) flags=0x0
      Gen(4), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: primary, disabled by unselected.
      Members:
        1: Seq_num(1 port1), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
    
    Service(2): Address Mode(IPV4) flags=0x0
      Gen(7), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: secondary
      Members:
        1: Seq_num(2 port2), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
    
  4. Verify neighbor routers:
    1. Primary neighbor router:
      FGT-NBR1 # get router info bgp network 192.168.20.0
      BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        64512
          10.100.1.2 from 10.100.1.2 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:5
            Last update: Thu Apr 30 15:41:58 2020
      
    2. Secondary neighbor router:
      FGT-NBR2 # get router info bgp network 192.168.20.0
      VRF 0 BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        Original VRF 0
        64512
          10.100.1.6 from 10.100.1.6 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:2
            Last update: Thu Apr 30 15:42:07 2020
      

Controlling traffic with BGP route mapping and service rules

Controlling traffic with BGP route mapping and service rules

SD-WAN allows you to select different outbound WAN links based on performance SLAs. It is important that BGP neighbors are aware of these settings, and changes to them.

BGP can adapt to changes in SD-WAN link SLAs in the following ways:

  • Applying different route-maps based on the SD-WAN's health checks. For example, different BGP community strings can be advertised to BGP neighbors when SLAs are not met.
  • Traffic can be selectively forwarded based on the active BGP neighbor. If the SD-WAN service's role matches the active SD-WAN neighbor, the service is enabled. If there is no match, then the service is disabled.

Example

In this topology, a branch FortiGate has two SD-WAN gateways serving as the primary and secondary gateways. The gateways reside in different datacenters, but have a full mesh network between them.

This example shows how route-maps and service rules are selected based on performance SLAs and the member that is currently active. Traffic flows through the primary gateway unless the neighbor's health check is outside of its SLA. If that happens, traffic routes to the secondary gateway.

BGP NBR1 is the primary neighbor and BGP NBR2 is the secondary neighbor.

The branch FortiGate's wan1 and wan2 interfaces are members of the SD-WAN. When the SD-WAN neighbor status is primary, it will advertise community 20:1 to BGP NBR1 and 20:5 to BGP NBR2. When the SD-WAN neighbor status is secondary, it will advertise 20:5 to BGP NBR1 and 20:2 to BGP NBR2.

Only one of the primary or secondary neighbors can be active at one time. The SD-WAN neighbor status is used to decide which neighbor is selected:

  • Primary: The primary neighbor takes precedence if its SLAs are met.
  • Secondary: If the primary neighbor's SLAs are not met, the secondary neighbor becomes active if its SLAs are met.
  • Standalone: If neither the primary or secondary neighbor's SLAs are met, the SD-WAN neighbor status becomes standalone.

Route map

SD-WAN is configured to let BGP advertise different communities when the SLA status changes. When the SLA is missed, it triggers BGP to advertise a different community to its BGP neighbor based on its route-map. The BGP neighbors can use the received community string to select the best path to reach the branch.

To configure BGP route-maps and neighbors:
  1. Configure an access for the routes to be matched:
    config router access-list
        edit "net192"
            config rule
                edit 1
                    set prefix 192.168.20.0 255.255.255.0
                next
            end
        next
    end
  2. Configure the primary neighbor's preferred route-map:
    config router route-map
        edit "comm1"
            config rule
                edit 1
                    set match-ip-address "net192"
                    set set-community "20:1"
                next
            end
        next
    end
    
  3. Configure the secondary neighbor's preferred route-map:
    config router route-map
        edit "comm2"
            config rule
                edit 1
                    set match-ip-address "net192"
                    set set-community "20:2"
                next
            end
        next
    end
    
  4. Configure the failed route-map:
    config router route-map
        edit "comm5"
            config rule
                edit 1
                    set match-ip-address "net192"
                    set set-community "20:5"
                next
            end
        next
    end
    
  5. Configure BGP neighbors:
    config router bgp
        set as 65412
        set router-id 1.1.1.1
        set ibgp-multipath enable
        config neighbor
            edit "10.100.1.1"
                set soft-reconfiguration enable
                set remote-as 20
                set route-map-out "comm5"
                set route-map-out-preferable "comm1"
            next
            edit "10.100.1.5"
                set soft-reconfiguration enable
                set remote-as 20
                set route-map-out "comm5"
                set route-map-out-preferable "comm2"
            next
        end
    end

    When SLAs are met, route-map-out-preferable is used. When SLAs are missed, route-map-out is used.

To configure SD-WAN:
  1. Configure the SD-WAN members:
    config system sdwan
        set status enable
        config members
            edit 1
                set interface "port1"
            next
            edit 2
                set interface "port2"
            next
        end
    end
  2. Configure health checks for each member:
    config system sdwan
        config health-check
            edit "ping"
                set server "10.100.2.22"
                set members 1
                config sla
                    edit 1
                        set link-cost-factor packet-loss
                        set packetloss-threshold 1
                    next
                end
            next
            edit "ping2"
                set server "10.100.2.23"
                set members 2
                config sla
                    edit 1
                        set link-cost-factor packet-loss
                        set packetloss-threshold 1
                    next
                end
            next
        end
    end
  3. Configure the SD-WAN neighbors and assign them a role and the health checks used to determine if the neighbor meets the SLA:

    SD-WAN neighbors can only be configured in the CLI.

    config system sdwan
        config neighbor
            edit "10.100.1.1"
                set member 1
                set role primary
                set health-check "ping"
                set sla-id 1
            next
            edit "10.100.1.5"
                set member 2
                set role secondary
                set health-check "ping2"
                set sla-id 1
            next
        end
    end

Service rules

Create SD-WAN service rules to direct traffic to the primary neighbor when its SLAs are met, and to the secondary neighbor when the primary neighbor's SLAs are missed.

To configure the SD-WAN service rules:
config system sdwan
    config service
        edit 1
            set name "Primary-Out"
            set role primary
            set dst "all"
            set src "all"
            set priority-members 1
        next
        edit 2
            set name "Secondary-Out"
            set role secondary
            set dst "all"
            set src "all"
            set priority-members 2
        next
    end
end
Tooltip

If neither the primary nor secondary neighbors are active, the SD-WAN neighbor status becomes standalone. Only service rules with standalone-action enabled will continue to pass traffic. This option is disabled by default.

Verification

To verify when the primary neighbor is passing traffic:
  1. Verify the health check status:
    FortiGate-Branch # diagnose sys sdwan health-check
    Health Check(ping):
    Seq(1 port1): state(alive), packet-loss(0.000%) latency(0.569), jitter(0.061) sla_map=0x1
    Health Check(ping2):
    Seq(2 port2): state(alive), packet-loss(0.000%) latency(3.916), jitter(2.373) sla_map=0x1
    
  2. Verify SD-WAN neighbor status:
    FortiGate-Branch # diagnose sys sdwan neighbor
    SD-WAN neighbor status: hold-down(disable), hold-down-time(0), hold_boot_time(0)
            Selected role(primary) last_secondary_select_time/current_time in seconds 0/572
    Neighbor(10.100.1.1): member(1) role(primary)
            Health-check(ping:1)  sla-pass selected alive
    Neighbor(10.100.1.5): member(2) role(secondary)
            Health-check(ping2:1)  sla-pass alive
  3. Verify service rules status:
    FortiGate-Branch # diagnose sys sdwan service
    
    Service(1): Address Mode(IPV4) flags=0x0
      Gen(3), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: primary
      Members:
        1: Seq_num(1 port1), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
    
    Service(2): Address Mode(IPV4) flags=0x0
      Gen(6), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: secondary, disabled by unselected.
      Members:
        1: Seq_num(2 port2), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
  4. Verify neighbor routers:
    1. Primary neighbor router:
      FGT-NBR1 # get router info bgp network 192.168.20.0
      BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        64512
          10.100.1.2 from 10.100.1.2 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:1
            Last update: Thu Apr 30 13:41:40 2020
    2. Secondary neighbor router:
      FGT-NBR2 # get router info bgp network 192.168.20.0
      VRF 0 BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        Original VRF 0
        64512
          10.100.1.6 from 10.100.1.6 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:5
            Last update: Thu Apr 30 13:41:39 2020
      
To verify when the secondary neighbor is passing traffic:
  1. Verify the health check status:
    FortiGate-Branch # diagnose sys sdwan health-check
    Health Check(ping):
    Seq(1 port1): state(dead), packet-loss(54.000%) sla_map=0x0
    Health Check(ping2):
    Seq(2 port2): state(alive), packet-loss(0.000%) latency(4.339), jitter(3.701) sla_map=0x1
  2. Verify SD-WAN neighbor status:
    FortiGate-Branch # diagnose sys sdwan neighbor
    SD-WAN neighbor status: hold-down(disable), hold-down-time(0), hold_boot_time(0)
            Selected role(secondary) last_secondary_select_time/current_time in seconds 936/936
    Neighbor(10.100.1.1): member(1) role(primary)
            Health-check(ping:1)  sla-fail dead
    Neighbor(10.100.1.5): member(2) role(secondary)
            Health-check(ping2:1)  sla-pass selected alive
  3. Verify service rules status:
    FortiGate-Branch # diagnose sys sdwan service
    
    Service(1): Address Mode(IPV4) flags=0x0
      Gen(4), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: primary, disabled by unselected.
      Members:
        1: Seq_num(1 port1), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
    
    Service(2): Address Mode(IPV4) flags=0x0
      Gen(7), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual)
      Service role: secondary
      Members:
        1: Seq_num(2 port2), alive, selected
      Src address:
            0.0.0.0-255.255.255.255
    
      Dst address:
            0.0.0.0-255.255.255.255
    
  4. Verify neighbor routers:
    1. Primary neighbor router:
      FGT-NBR1 # get router info bgp network 192.168.20.0
      BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        64512
          10.100.1.2 from 10.100.1.2 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:5
            Last update: Thu Apr 30 15:41:58 2020
      
    2. Secondary neighbor router:
      FGT-NBR2 # get router info bgp network 192.168.20.0
      VRF 0 BGP routing table entry for 192.168.20.0/24
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Not advertised to any peer
        Original VRF 0
        64512
          10.100.1.6 from 10.100.1.6 (192.168.122.98)
            Origin IGP metric 0, localpref 100, valid, external, best
            Community: 20:2
            Last update: Thu Apr 30 15:42:07 2020