Fortinet white logo
Fortinet white logo

BGP routing

BGP routing

NOTE: You must have an advanced features license to use BGP routing.

Border Gateway Protocol (BGP) contains two distinct subsets: internal BGP (iBGP) and external BGP (eBGP). iBGP is intended for use within your own networks. eBGP is used to connect many different networks together and is the main routing protocol for the Internet backbone. FortiSwitch units support iBGP, and eBGP only for communities.

BGP was first used in 1989. The current version, BGP-4, was released in 1995 and is defined in RFC 1771. That RFC has since been replaced by RFC 4271. The main benefits of BGP-4 are classless inter-domain routing and aggregate routes. BGP is the only routing protocol to use TCP for a transport protocol. Other routing protocols use UDP.

BGP makes routing decisions based on path, network policies, and rulesets instead of the hop-count metric as RIP does, or cost-factor metrics as OSPF does.

BGP-4+ supports IPv6. It was introduced in RFC 2858 and RFC 2545.

BGP is the routing protocol used on the Internet. It was designed to replace the old Exterior Gateway Protocol (EGP) which had been around since 1982, and was very limited. BGP enabled more networks to take part in the Internet backbone to effectively decentralize it and make the Internet more robust, and less dependent on a single ISP or backbone network.

Parts and terminology of BGP

In a BGP network, there are some terms that need to be explained before going ahead. Some parts of BGP are not explained here because they are common to other dynamic routing protocols. When determining your network topology, note that the number of available or supported routes is not set by the configuration but depends on the available memory on the FortiSwitch units.

BGP and IPv6

FortiSwitch units support IPv6 over BGP using the same config router bgp CLI command as IPv4 but different subcommands.

The main CLI keywords have IPv6 equivalents that are identified by the “6” on the end of the keyword, such as config network6 or set allowas-in6. For more information about IPv6 BGP keywords, see the FortiSwitchOS CLI Reference.

Role of routers in BGP networks

Dynamic routing has a number of different roles that routers can fill. BGP has a number of custom roles that routers can fill. These include speaker routers, peer routers or neighbors, and route reflectors.

Speaker routers

Any router that is configured for BGP is considered a BGP speaker. This means that a speaker router advertises BGP routes to its peers.

Any routers on the network that are not speaker routers are not treated as BGP routers.

Peer routers or neighbors

In a BGP network, all neighboring BGP routers or peer routers are routers that are connected to a FortiSwitch unit. A FortiSwitch unit learns about all other routers through these peers.

You need to manually configure BGP peers on a FortiSwitch unit as neighbors. Otherwise, these routers are not seen as peers but simply as other routers on the network that do not support BGP. Optionally, you can use MD5 authentication to password-protect BGP sessions with those neighbors (see RFC 2385).

You can configure up to 1000 BGP neighbors on a FortiSwitch unit. You can clear all or some BGP neighbor connections (sessions), using the execute router clear bgp CLI command.

For example, if you have 10 routes in the BGP routing table and you want to clear the specific route to IP address 10.10.10.1, enter the following CLI command:

execute router clear bgp ip 10.10.10.1

To remove all routes for AS number 650001, enter the following CLI command:

execute router clear bgp as 650001

To remove route flap dampening information for the 10.10.0.0/16 subnet, enter the following CLI command:

execute router clear bgp dampening 10.10.0.0/16

In the following diagram, Router A is directly connected to five other routers in a network that contains 12 routers. These routers (the ones in the blue circle) are Router A’s peers or neighbors.

Router A and its five peer routers

As a minimum, when configuring BGP neighbors, you must enter their IP address and the AS number (remote-as). This is all of the information the GUI allows you to enter for a neighbor.

The following BGP commands are related to neighbors:

config router bgp

config neighbor

edit "<IPv4_IPv6_address>"

set advertisement-interval <0-600>

set allowas-in-enable {disable | enable}

set allowas-in <1-10>

set allowas-in-enable6 {disable | enable}

set allowas-in6 <1-10>

set attribute-unchanged {as-path | MED | next-hop}

set attribute-unchanged6 {as-path | MED | next-hop}

set activate {disable | enable}

set activate6 {disable | enable}

set bfd {disable | enable}

set capability-dynamic {disable | enable}

set capability-orf {both | none | receive | send}

set capability-orf6 {both | none | receive | send}

set capability-default-originate {disable | enable}

set capability-default-originate6 {disable | enable}

set dont-capability-negotiate {disable | enable}

set ebgp-enforce-multihop {disable | enable}

set ebgp-multihop-ttl <1-255>

set ebgp-ttl-security-hops <1-254>

set next-hop-self {disable | enable}

set next-hop-self6 {disable | enable}

set override-capability {disable | enable}

set passive {disable | enable}

set remove-private-as {disable | enable}

set remove-private-as6 {disable | enable}

set route-reflector-client {disable | enable}

set route-reflector-client6 {disable | enable}

set route-server-client {disable | enable}

set route-server-client6 {disable | enable}

set shutdown {disable | enable}

set soft-reconfiguration {disable | enable}

set soft-reconfiguration6 {disable | enable}

set as-override {disable | enable}

set as-override6 {disable | enable}

set strict-capability-match {disable | enable}

set description <string>

set distribute-list-in <string>

set distribute-list-in6 <string>

set distribute-list-out <string>

set distribute-list-out6 <string>

set filter-list-in <string>

set filter-list-in6 <string>

set filter-list-out <string>

set filter-list-out6 <string>

set interface <interface_name>

set maximum-prefix <1-4294967295>

set maximum-prefix6 <1-4294967295>

set prefix-list-in <string>

set prefix-list-in6 <string>

set prefix-list-out <string>

set prefix-list-out6 <string>

set remote-as <MANDATORY_1-4294967295>

set route-map-in <string>

set route-map-in6 <string>

set route-map-out <string>

set route-map-out6 <string>

set send-community {both | disable | extended | standard}

set send-community6 {both | disable | extended | standard}

set keep-alive-timer <0-65535>

set holdtime-timer <0, 3-65535>

set connect-timer <0-65535>

set unsuppress-map <string>

set unsuppress-map6 <string>

set update-source {interface_name}

set weight <0-65535>

end

end

end

Route reflectors

Route reflectors (RR) in BGP concentrate route updates so other routers only need to talk to the RRs to get all of the updates. This results in smaller routing tables, fewer connections between routers, faster responses to network topology changes, and less administration bandwidth. BGP RRs are defined in RFC 1966.

In a BGP RR configuration, the AS is divided into different clusters that each include client and reflector routers. The client routers supply the reflector routers with the client’s route updates. The reflectors pass this information along to other RRs and border routers. Only the reflectors need to be configured, not the clients, because the clients find the closest reflector and communicate with it automatically. The reflectors communicate with each other as peers. A FortiSwitch unit can be configured as either reflectors or clients.

Because RRs are processing more than the client routers, the reflectors should have more resources to handle the extra workload.

Smaller networks running BGP typically do not require RRs. However, RRs are a useful feature for large companies, where their AS may include 100 routers or more. For example, a full mesh 20 router configuration within an AS, there would have to be 190 unique BGP sessions just for routing updates within the AS. The number of sessions jumps to 435 sessions for just 30 routers, or 4950 sessions for 100 routers. Based on these numbers, updating this many sessions will quickly consume the limited bandwidth and processing resources of the routers involved.

The following diagram illustrates how RRs can improve the situation when only six routers are involved. The AS without RRs requires 15 sessions between the routers. In the AS with RRs, the two RRs receive route updates from the reflector clients (unlabeled routers in the diagram) in their cluster, as well as other RRs, and pass them on to the border router. The RR configuration requires only six sessions. This example shows a reduction of 60% for the number of required sessions.

Required sessions within an AS with and without RRs

The BGP commands related to RRs include:

config router bgp

config neighbor

edit "<IPv4_IPv6_address>"

set route-reflector-client {disable | enable}

set route-reflector-client6 {disable | enable}

set route-server-client {disable | enable}

set route-server-client6 {disable | enable}

end

end

Confederations

Confederations were introduced to reduce the number of BGP advertisements on a segment of the network and reduce the size of the routing tables. Confederations essentially break up an AS into smaller units. Confederations are defined in RFC 3065 and RFC 1965.

Within a confederation, all routers communicate with each other in a full mesh arrangement. Communications between confederations is more like inter-AS communications because many of the attributes are changed as they would be for BGP communications leaving the AS, or eBGP.

Confederations are useful when merging ASs. Each AS being merged can easily become a confederation, which requires few changes. Any additional permanent changes can then be implemented over time, as required. The diagram below shows the group of ASs before merging and the corresponding confederations afterward, as part of the single AS with the addition of a new border router. It should be noted that after merging, if the border router becomes a route reflector, then each confederation only needs to communicate with one other router instead of five others.

Confederations and RRs perform similar functions: they both sub-divide large ASs for more efficient operation. They differ in that route reflector clusters can include routers that are not members of a cluster, whereas routers in a confederation must belong to that confederation. Also, confederations place their confederation numbers in the AS_PATH attribute, making it easier to trace.

NOTE: While confederations essentially create sub-ASs, all the confederations within an AS appear as a single AS to external ASs.

Confederation related BGP commands include the following:

config router bgp

set confederation-identifier <peerid_integer>

end

Network Layer Reachability Information

Network Layer Reachability Information (NLRI) is unique to BGP-4. It is sent as part of the update messages sent between BGP routers and contains information necessary to supernet, or aggregate route, information. The NLRI includes the length and prefix that, when combined, are the address of the aggregated routes referred to.

There is only one NLRI entry per BGP update message.

BGP attributes

Each route in a BGP network has a set of attributes associated with it. These attributes define the route and are modified, as required, along the route.

BGP can work well with mostly default settings, but if you're going to change settings you need to understand the roles of each attribute and how they affect those settings.

The BGP attributes include the ones listed in the following table.

Attribute

Description

AS_PATH

A list of ASs a route has passed through. For more information, see AS_PATH.

MULTI_EXIT_DESC (MED)

Which router to use to exit an AS with more than one external connection. For more information, see MULTI_EXIT_DESC.

COMMUNITY

Used to apply attributes to a group of routes. For more information, see COMMUNITY.

NEXT_HOP

Where the IP packets should be forwarded to, like a gateway in static routing. For more information, see NEXT_HOP.

ATOMIC_AGGREGATE

Used when routes have been summarized to tell downstream routers not to de-aggregate the route. For more information, see ATOMIC_AGGREGATE.

ORIGIN

Used to determine if the route is from the local AS or not. For more information, see ORIGIN.

LOCAL_PREF

Used only within an AS to select the best route to a location (like MED).

Inbound policies on FortiSwitch units can change the NEXT-HOP, LOCAL-PREF, MED, and AS-PATH attributes of an internal BGP (iBGP) route for its local route selection purposes. However, outbound policies on the device cannot affect these attributes.

AS_PATH

AS_PATH is the BGP attribute that keeps track of each AS that a route advertisement has passed through. AS_PATH is used by confederations and by exterior BGP (EBGP) to help prevent routing loops. A router knows there is a loop if it receives an AS_PATH with that router's AS in it. The diagram shows the route between Router A and Router B. The AS_PATH from A to B would read 701,702,703 for each AS that the route passes through.

As of the beginning of 2010, the industry upgraded from 2-byte to 4-byte AS_PATHs. This upgrade was due to the imminent exhaustion of 2-byte AS_PATH numbers. FortiOS supports 4-byte AS_PATHs in its BGP implementation.

AS_PATH of 701,702, 703 between routers A and B

The BGP commands related to AS_PATH include the following:

config router bgp

set bestpath-as-path-ignore {enable | disable}

end

MULTI_EXIT_DESC

BGP AS systems can have one or more routers that connect them to other ASs. For ASs with more than one connecting router, the Multi-Exit Discriminator (MED) lists which router is best to use when leaving the AS. The MED is based on attributes, such as delay. It is a recommendation only, as some networks may have different priorities.

BGP updates advertise the best path to a destination network. When a FortiSwitch unit receives a BGP update, the FortiSwitch unit examines the MED attribute of potential routes to determine the best path to a destination network before recording the path in the local FortiSwitch routing table.

FortiSwitch units have the option to treat any routes without an MED attribute as the worst possible routing choice. This can be useful because a lack of MED information is a lack of routing information, which can be suspicious as a possible hacking attempt or an attack on the network. At best, it signifies an unreliable route to select.

The BGP commands related to MED include the following:

config router bgp

set always-compare-med {enable | disable}

set bestpath-med-confed {enable | disable}

set bestpath-med-missing-as-worst {enable | disable}

set deterministic-med {enable | disable}

config neighbor

edit "<IPv4_IPv6_address>"

set attribute-unchanged [as-path] [med] [next-hop]

set attribute-unchanged6 {as-path | MED | next-hop}

end

end

end

COMMUNITY

A community is a group of routes that have the same routing policies applied to them. This saves time and resources. A community is defined by the COMMUNITY attribute of a BGP route.

A FortiSwitch unit can set the COMMUNITY attribute of a route to assign the route to predefined paths (see RFC 1997). The FortiSwitch unit can examine the COMMUNITY attribute of learned routes to perform local filtering and/or redistribution.

The BGP commands related to COMMUNITY include the following:

config router bgp

set send-community {both | disable | extended | standard}

set send-community6 {both | disable | extended | standard}

end

NEXT_HOP

The NEXT_HOP attribute says what IP address the packets should be forwarded to next. Each time the route is advertised, this value is updated. The NEXT_HOP attribute is much like a gateway in static routing.

FortiSwitch units allow you to change the advertising of the FortiSwitch unit’s IP address (instead of the neighbor’s IP address) in the NEXT_HOP information that is sent to IBGP peers. This is changed with the config neighbor, set next-hop-self command.

The BGP commands related to NEXT_HOP include the following:

config router bgp

config neighbor

edit "<IPv4_IPv6_address>"

set attribute-unchanged [as-path] [med] [next-hop]

set attribute-unchanged6 {as-path | MED | next-hop}

set next-hop-self {enable | disable}

set next-hop-self6 {disable | enable}

next

end

end

ATOMIC_AGGREGATE

The ATOMIC_AGGREGATE attribute is used when routes have been summarized. It indicates which AS and which router summarize the routes. It also tells downstream routers not to de-aggregate the route. Summarized routes are routes with similar information that have been combined, or aggregated, into one route that is easier to send in updates for. When it reaches its destination, the summarized routes are split back up into the individual routes.

The FortiSwitch unit does not specifically set this attribute in the BGP router command, but it is used in the route map command.

The CLI commands related to ATOMIC_AGGREGATE include the following:

config router route-map

edit <route_map_name>

set protocol bgp

config rule

edit <route_map_rule_id>

set set-aggregator-as <id_integer>

set set-aggregator-ip <address_ipv4>

set set-atomic-aggregate {enable | disable}

end

end

end

ORIGIN

The ORIGIN attribute records where the route came from. The options can be IBGP, EBGP, or incomplete. This information is important because internal routes (IBGP) are, by default, higher priority than external routes (EBGP). However, incomplete ORIGINs are the lowest priority of the three.

The CLI commands related to ORIGIN include the following:

config router route-map

edit <route_map_name>

set protocol bgp

config rule

edit <route_map_rule_id>

set match-origin {egp | igp | incomplete | none}

end

end

end

How BGP works

BGP is a link-state routing protocol and keeps link-state information about the status of each network link it has connected. A BGP router receives information from its peer routers that have been defined as neighbors. BGP routers listen for updates from these configured neighboring routers on TCP port 179.

A BGP router is a finite state machine with six various states for each connection. As two BGP routers discover each other and establish a connection, they go from the idle state and through the various states until they reach the established state. An error can cause the connection to drop and the state of the router to reset to either active or idle. These errors can be caused by TCP port 179 not being open, a random TCP port above port 1023 not being open, the peer address being incorrect, or the AS number being incorrect.

When BGP routers start a connection, they negotiate which (if any) optional features will be used, such as multiprotocol extensions, that can include IPv6 and VPNs.

IBGP versus EBGP

When you read about BGP, you often see EBGP or IBGP mentioned. These are both BGP routing, but BGP used in different roles. Exterior BGP (EBGP) involves packets crossing multiple autonomous systems (ASs) and interior BGP (IBGP) involves packets that stay within a single AS. For example, the AS_PATH attribute is only useful for EBGP where routes pass through multiple ASs.

These two modes are important because some features of BGP are used only for one of EBGP or IBGP. For example, confederations are used in EBGP and RRs are used only in IBGP. Also, routes learned from IBGP have priority over routes learned from EBGP.

FortiSwitch units have some commands that are specific to EBGP, including the following:

  • automatically resetting the session information to external peers if the connection goes down:set fast-external-failover {enable | disable}
  • setting an administrative distance for all routes learned from external peers (you must also configure local and internal distances if this is set):set distance-external <distance_integer>
  • enforcing EBGP multihops and their TTL (number of hops): set ebgp-enforce-multihop {enable | disable} and set ebgp-multihop-ttl <seconds_integer>

BGP path determination: Which route to use

Firstly, recall that the number of available or supported routes is not set by the configuration but depends on the available memory on the FortiSwitch unit. All learned routes and their attributes come into the BGP router in raw form. Before routes are installed in the routing table or are advertised to other routers, three levels of decisions must be made.

The three phases of BGP best path determination do not change. However, some manufacturers have added more information to the process, such as Cisco’s WEIGHT attribute, to allow an administrator to force one route’s selection over another.

There is one Adj-RIB-IN and Adj-RIB-OUT for each configured neighbor. They are updated when the FortiSwitch unit receives BGP updates or when the FortiSwitch unit sends out BGP updates.

The three phases of a BGP routing decision

Decision phase 1

At this phase, the decision is to calculate how preferred each route and its NRLI are the Adjacent Routing Information Base Incoming (Adj-RIBs-In) compared to the other routes. For internal routes (IBGP), policy information or LOCAL_PREF is used. For external peer learned routes, it is based strictly on policy. These rules set up a list of which routes are most preferred going into Phase 2.

Decision phase 2

Phase 2 involves installing the best route to each destination into the local Routing Information Base (Loc-RIB). Effectively, the Loc-RIB is the primary routing table. Each route from Phase 1 has their NEXT_HOP checked to ensure the destination is reachable. If it is reachable, the AS_PATH is checked for loops. After that, routes are installed based on the following decision process:

  • If there is only one route to a location, it is installed.
  • If there are multiple routes to the same location, use the most preferred route from Level 1.
  • If there is a tie, break the tie based on the following, in descending order of importance: shortest AS_PATH, smallest ORIGIN number, smallest MED, EBGP over IBGP, smallest metric or cost for reaching the NEXT_HOP, BGP identifier, and lowest IP address.

Note that the new routes that are installed into the Loc-RIB are in addition to any existing routes in the table. Once Phase 2 is completed, the Loc-RIB will consist of the best of both the new and older routes.

Decision phase 3

Phase 3 is route distribution or dissemination. This is the process of deciding which routes the router will advertise. If there is any route aggregation or summarizing, it happens here. Also, any route filtering from route maps happens here.

Once Phase 3 is complete, an update can be sent out to update the neighbor of new routes.

Aggregate routes and addresses

BGP-4 allows classless routing, which uses netmasks as well as IP addresses. This classless routing allows the configuration of aggregate routes by stating the address bits the aggregated addresses have in common.

The ATOMIC_AGGREGATE attribute informs routers that the route has been aggregated and should not be de-aggregated. An associated AGGREGATOR attribute include the information about the router that did the aggregating including its AS.

The BGP commands associated with aggregate routes and addresses are the following:

config router bgp

config aggregate-address

edit <aggr_addr_id>

set as-set {enable | disable}

set prefix <address_ipv4mask>

set summary-only {enable | disable}

end

end

config aggregate-address6

edit <aggr_addr_id>

set as-set {enable | disable}

set prefix6 <address_ipv6mask>

set summary-only {enable | disable}

end

end

Troubleshooting BGP

There are some features in BGP that are used to deal with problems that may arise. Typically, the problems with a BGP network that has been configured involve routes going offline frequently. This is called route flap and causes problems for the routers using that route.

Clearing routing table entries

To see if a new route is being properly added to the routing table, you can clear all or some BGP neighbor connections (sessions) using the execute router clear bgp command.

For example, if you have 10 routes in the BGP routing table and you want to clear the specific route to IP address 10.10.10.1, enter the following CLI command:

execute router clear bgp ip 10.10.10.1

To remove all routes for AS number 650001, enter the following CLI command:

execute router clear bgp as 650001

Route flap

When routers or hardware along a route go offline and back online that is called a route flap. Flapping is the term that is used if these outages continue, especially if they occur frequently.

Route flap is a problem in BGP because each time a peer or a route goes down, all the peer routers that are connected to that out-of-service router advertise the change in their routing tables. This creates a lot of administration traffic on the network and the same traffic re-occurs when that router comes back online. If the problem is something like a faulty network cable that wobbles online and offline every 10 seconds, there could easily be an overwhelming amount of routing updates sent out unnecessarily.

Another possible reason for route flap occurs with multiple FortiSwitch units in HA mode. When an HA cluster fails over to the secondary unit, other routers on the network may see the HA cluster as being offline, resulting in route flap. While this does not occur often, or more than once at a time, it can still result in an interruption in traffic that is unpleasant for network users. The easy solution for this problem is to increase the timers on the HA cluster, such as TTL timers, so they do not expire during the failover process. Also, configuring graceful restart on the HA cluster helps with a smooth failover.

The first method of dealing with route flap is to check your hardware. If a cable is loose or bad, it can easily be replaced and eliminate the problem. If an interface on the router is bad, either avoid using that interface or swap in a functioning router. If the power source is bad on a router, either replace the power supply or use a power conditioning backup power supply. These quick and easy fixes can save you from configuring more complex BGP options. However, if the route flap is from another source, configuring BGP to deal with the outages will ensure your network users uninterrupted service.

Some methods of dealing with route flap in BGP include:

Holdtime timer

The first line of defense to a flapping route is the holdtime timer. This timer reduces how frequently a route going down will cause a routing update to be broadcast.

After it is activated, the holdtime timer does not allow the FortiSwitch unit to accept any changes to that route for the duration of the timer. If the route flaps five times during the timer period, only the first outage is recognized by the FortiSwitch unit. For the duration of the other outages, there will not be changes because the FortiSwitch unit is essentially treating this router as down. If the route is still flapping after the timer expires, it'll happen all over again.

Even if the route is not flapping (for example, if it goes down, comes up, and stays back up) the timer still counts down and the route is ignored for the duration of the timer. In this situation, the route is seen as down longer than it really is but there will be only the one set of route updates. This is not a problem in normal operation because updates are not frequent.

Also, the potential for a route to be treated as down when it is really up can be viewed as a robustness feature. Typically, you do not want most of your traffic being routed over an unreliable route. So if there is route flap going on, it is best to avoid that route if you can. This is enforced by the holdtime timer.

How to configure the holdtime timer

There are three different route flapping situations that can occur: the route goes up and down frequently, the route goes down and back up once over a long period of time, or the route goes down and stays down for a long period of time. These can all be handled using the holdtime timer.

For example, your network has two routes that you want to set the timer for. One is your main route (to 10.12.101.4) that all of your Internet traffic goes through, and it cannot be down for long if it is down. The second is a low speed connection to a custom network that is used infrequently (to 10.13.101.4). The timer for the main route should be fairly short (for example, 60 seconds). The second route timer can be left at the default because it is rarely used. In your BGP configuration, this looks like the following:

config router bgp

config neighbor

edit 10.12.101.4

set holdtime-timer 60

next

edit 10.13.101.4

set holdtime-timer 180

next

end

end

Dampening

Dampening is a method that is used to limit the amount of network problems due to flapping routes. With dampening, the flapping still occurs but the peer routers pay less and less attention to that route as it flaps more often. One flap does not start dampening, but the second flap starts a timer where the router will not use that route because it is considered unstable. If the route flaps again before the timer expires, the timer continues to increase. There is a period of time called the reachability half-life, after which a route flap will be suppressed for only half the time. This half-life comes into effect when a route has been stable for a while but not long enough to clear all the dampening completely. For the flapping route to be included in the routing table again, the suppression time must expire.

If the route flapping was temporary, you can clear the flapping or dampening from the FortiSwitch unit's cache by using one of the execute router clear bgp CLI commands:

execute router clear bgp dampening {<ip_address> | <ip/netmask>}

For example, to remove route flap dampening information for the 10.10.0.0/16 subnet, enter the following CLI command:

execute router clear bgp dampening 10.10.0.0/16

The BGP commands related to route dampening are the following:

config router bgp

set dampening {enable | disable}

set dampening-max-suppress-time <minutes_integer>

set dampening-reachability-half-life <minutes_integer>

set dampening-reuse <reuse_integer>

set dampening-suppress <limit_integer>

end

BFD

Bidirectional Forwarding Detection (BFD) is a protocol that you can use to quickly locate hardware failures in the network. Routers running BFD communicate with each other and if a timer runs out on a connection then that router is declared down. BFD then communicates this information to the routing protocol and the routing information is updated. For more information about BFD, see Bidirectional forwarding detection .

Configuring BGP

Configuring BGP on the FortiSwitch unit includes the following major steps:

  1. Enter the BGP configuration mode.
  2. Set the autonomous system and router identifier.
  3. Configure a BGP neighbor.
  4. Redistribute non-BGP routes. Advertise these non-BGP routes within BGP.

1. Enter the BGP configuration mode

Enter the BGP configuration mode to access all of the BGP configuration commands:

# config router bgp

2. Set the autonomous system and router identifier

Set the autonomous system. For IBGP, the AS value needs to match the remote-as value in the neighbor router. For EBGP, the AS value differs from the remote-as value in the neighbor router. You also need to specify a fixed router identifier for the FortiSwitch unit. These two commands are mandatory.

# set as <AS number>

# set router-id <IP_address>

3. Configure the BGP neighbors

Configure the BGP neighbors.

NOTE: For IBGP, if the IP address of the BGP neighbor is a loopback address, you must use the set update-source cmd command to specify which interface address will be used as the source IP address in the outgoing BGP packet.

config neighbor

edit "<IPv4_or_IPv6 address>"

set remote-as <1-4294967295>

end

4. Redistribute non-BGP routes

Redistribute non-BGP IPv4 or IPv6 routes within BGP:

config redistribute {connected | isis | ospf | rip | static}

set status enable

set route-map <string>

end

config redistribute6 {connected | isis | ospf | rip | static}

set status {disable | enable}

set route-map <string>

end

Other BGP commands

Clearing the BGP routes

Use the following commands to clear the BGP routes:

execute router clear bgp all

execute router clear bgp ip <IPv4_or_IPv6_address>

execute router clear bgp ipv6 <IPv4_or_IPv6_address>

execute router clear bgp as <AS_number>

execute router clear bgp dampening <IP_address>

Checking the BGP configuration

The get router info bgp and get router info6 bgp commands have options to display different aspects of the BGP configuration and status.

For example:

get router info bgp neighbors

get router info bgp network

get router info6 bgp filter-list

get router info6 bgp route-map

Changing the maximum number of paths for ECMP

If you are using equal-cost multi-path (ECMP) routing with the EBGP or IBGP, the maximum number of paths is 1 by default. Use the following commands to change the default:

config router bgp

set maximum-paths-ebgp <1-64>

set maximum-paths-ibgp <1-64>

end

Sample configurations

Here is an example of a BGP routing configuration:

Configure system interfaces

Interface configuration for FortiSwitch 1:

config system interface

edit mgmt

set ip 10.105.7.9 255.255.255.0

set allowaccess ping https http ssh telnet

set type physical

next

edit internal

set type physical

next

edit vlan20-p2

set ip 192.168.2.100 255.255.255.0

set allowaccess ping https http ssh telnet

set vlanid 20

set interface internal

next

edit vlan40-p4

set ip 172.168.111.6 255.255.255.0

set allowaccess ping https http ssh telnet

set vlanid 40

set interface internal

end

config switch interface

edit "port2"

set native-vlan 20

set stp-state disabled

next

edit "port4"

set native-vlan 40

set stp-state disabled

next

edit "internal"

set allowed-vlans 1,20, 40, 4094

set stp-state disabled

next

end

Internal BGP

In this example, the two neighboring switches are in the same autonomous system.

Configuration for FortiSwitch 1:

config router bgp

set as 6500

set router-id 1.2.3.4

config neighbor

edit "172.168.111.5"

set remote-as 6500

next

end

config network

edit 1

set prefix 192.168.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

Configuration for FortiSwitch 2:

config router bgp

set as 6500

set router-id 5.6.7.8

config neighbor

edit "172.168.111.6"

set remote-as 6500

next

end

config network

edit 1

set prefix 10.50.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

External BGP

In this example, the two neighboring switches are in separate autonomous systems.

Configuration for FortiSwitch 1:

config router bgp

set as 6500

set router-id 1.2.3.4

config neighbor

edit "172.168.111.5"

set remote-as 7500

next

end

config network

edit 1

set prefix 192.168.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

Configuration for FortiSwitch 2:

config router bgp

set as 7500

set router-id 5.6.7.8

config neighbor

edit "172.168.111.6"

set remote-as 6500

next

end

config network

edit 1

set prefix 10.50.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

Using the following command, you can check the BGP status on the local switch:

# get router info bgp summary

To check the details about the BGP neighbors:

# get router info bgp neighbors

To check the routes learned by BGP, use the following command:

# get router info routing-table details

BGP routing

BGP routing

NOTE: You must have an advanced features license to use BGP routing.

Border Gateway Protocol (BGP) contains two distinct subsets: internal BGP (iBGP) and external BGP (eBGP). iBGP is intended for use within your own networks. eBGP is used to connect many different networks together and is the main routing protocol for the Internet backbone. FortiSwitch units support iBGP, and eBGP only for communities.

BGP was first used in 1989. The current version, BGP-4, was released in 1995 and is defined in RFC 1771. That RFC has since been replaced by RFC 4271. The main benefits of BGP-4 are classless inter-domain routing and aggregate routes. BGP is the only routing protocol to use TCP for a transport protocol. Other routing protocols use UDP.

BGP makes routing decisions based on path, network policies, and rulesets instead of the hop-count metric as RIP does, or cost-factor metrics as OSPF does.

BGP-4+ supports IPv6. It was introduced in RFC 2858 and RFC 2545.

BGP is the routing protocol used on the Internet. It was designed to replace the old Exterior Gateway Protocol (EGP) which had been around since 1982, and was very limited. BGP enabled more networks to take part in the Internet backbone to effectively decentralize it and make the Internet more robust, and less dependent on a single ISP or backbone network.

Parts and terminology of BGP

In a BGP network, there are some terms that need to be explained before going ahead. Some parts of BGP are not explained here because they are common to other dynamic routing protocols. When determining your network topology, note that the number of available or supported routes is not set by the configuration but depends on the available memory on the FortiSwitch units.

BGP and IPv6

FortiSwitch units support IPv6 over BGP using the same config router bgp CLI command as IPv4 but different subcommands.

The main CLI keywords have IPv6 equivalents that are identified by the “6” on the end of the keyword, such as config network6 or set allowas-in6. For more information about IPv6 BGP keywords, see the FortiSwitchOS CLI Reference.

Role of routers in BGP networks

Dynamic routing has a number of different roles that routers can fill. BGP has a number of custom roles that routers can fill. These include speaker routers, peer routers or neighbors, and route reflectors.

Speaker routers

Any router that is configured for BGP is considered a BGP speaker. This means that a speaker router advertises BGP routes to its peers.

Any routers on the network that are not speaker routers are not treated as BGP routers.

Peer routers or neighbors

In a BGP network, all neighboring BGP routers or peer routers are routers that are connected to a FortiSwitch unit. A FortiSwitch unit learns about all other routers through these peers.

You need to manually configure BGP peers on a FortiSwitch unit as neighbors. Otherwise, these routers are not seen as peers but simply as other routers on the network that do not support BGP. Optionally, you can use MD5 authentication to password-protect BGP sessions with those neighbors (see RFC 2385).

You can configure up to 1000 BGP neighbors on a FortiSwitch unit. You can clear all or some BGP neighbor connections (sessions), using the execute router clear bgp CLI command.

For example, if you have 10 routes in the BGP routing table and you want to clear the specific route to IP address 10.10.10.1, enter the following CLI command:

execute router clear bgp ip 10.10.10.1

To remove all routes for AS number 650001, enter the following CLI command:

execute router clear bgp as 650001

To remove route flap dampening information for the 10.10.0.0/16 subnet, enter the following CLI command:

execute router clear bgp dampening 10.10.0.0/16

In the following diagram, Router A is directly connected to five other routers in a network that contains 12 routers. These routers (the ones in the blue circle) are Router A’s peers or neighbors.

Router A and its five peer routers

As a minimum, when configuring BGP neighbors, you must enter their IP address and the AS number (remote-as). This is all of the information the GUI allows you to enter for a neighbor.

The following BGP commands are related to neighbors:

config router bgp

config neighbor

edit "<IPv4_IPv6_address>"

set advertisement-interval <0-600>

set allowas-in-enable {disable | enable}

set allowas-in <1-10>

set allowas-in-enable6 {disable | enable}

set allowas-in6 <1-10>

set attribute-unchanged {as-path | MED | next-hop}

set attribute-unchanged6 {as-path | MED | next-hop}

set activate {disable | enable}

set activate6 {disable | enable}

set bfd {disable | enable}

set capability-dynamic {disable | enable}

set capability-orf {both | none | receive | send}

set capability-orf6 {both | none | receive | send}

set capability-default-originate {disable | enable}

set capability-default-originate6 {disable | enable}

set dont-capability-negotiate {disable | enable}

set ebgp-enforce-multihop {disable | enable}

set ebgp-multihop-ttl <1-255>

set ebgp-ttl-security-hops <1-254>

set next-hop-self {disable | enable}

set next-hop-self6 {disable | enable}

set override-capability {disable | enable}

set passive {disable | enable}

set remove-private-as {disable | enable}

set remove-private-as6 {disable | enable}

set route-reflector-client {disable | enable}

set route-reflector-client6 {disable | enable}

set route-server-client {disable | enable}

set route-server-client6 {disable | enable}

set shutdown {disable | enable}

set soft-reconfiguration {disable | enable}

set soft-reconfiguration6 {disable | enable}

set as-override {disable | enable}

set as-override6 {disable | enable}

set strict-capability-match {disable | enable}

set description <string>

set distribute-list-in <string>

set distribute-list-in6 <string>

set distribute-list-out <string>

set distribute-list-out6 <string>

set filter-list-in <string>

set filter-list-in6 <string>

set filter-list-out <string>

set filter-list-out6 <string>

set interface <interface_name>

set maximum-prefix <1-4294967295>

set maximum-prefix6 <1-4294967295>

set prefix-list-in <string>

set prefix-list-in6 <string>

set prefix-list-out <string>

set prefix-list-out6 <string>

set remote-as <MANDATORY_1-4294967295>

set route-map-in <string>

set route-map-in6 <string>

set route-map-out <string>

set route-map-out6 <string>

set send-community {both | disable | extended | standard}

set send-community6 {both | disable | extended | standard}

set keep-alive-timer <0-65535>

set holdtime-timer <0, 3-65535>

set connect-timer <0-65535>

set unsuppress-map <string>

set unsuppress-map6 <string>

set update-source {interface_name}

set weight <0-65535>

end

end

end

Route reflectors

Route reflectors (RR) in BGP concentrate route updates so other routers only need to talk to the RRs to get all of the updates. This results in smaller routing tables, fewer connections between routers, faster responses to network topology changes, and less administration bandwidth. BGP RRs are defined in RFC 1966.

In a BGP RR configuration, the AS is divided into different clusters that each include client and reflector routers. The client routers supply the reflector routers with the client’s route updates. The reflectors pass this information along to other RRs and border routers. Only the reflectors need to be configured, not the clients, because the clients find the closest reflector and communicate with it automatically. The reflectors communicate with each other as peers. A FortiSwitch unit can be configured as either reflectors or clients.

Because RRs are processing more than the client routers, the reflectors should have more resources to handle the extra workload.

Smaller networks running BGP typically do not require RRs. However, RRs are a useful feature for large companies, where their AS may include 100 routers or more. For example, a full mesh 20 router configuration within an AS, there would have to be 190 unique BGP sessions just for routing updates within the AS. The number of sessions jumps to 435 sessions for just 30 routers, or 4950 sessions for 100 routers. Based on these numbers, updating this many sessions will quickly consume the limited bandwidth and processing resources of the routers involved.

The following diagram illustrates how RRs can improve the situation when only six routers are involved. The AS without RRs requires 15 sessions between the routers. In the AS with RRs, the two RRs receive route updates from the reflector clients (unlabeled routers in the diagram) in their cluster, as well as other RRs, and pass them on to the border router. The RR configuration requires only six sessions. This example shows a reduction of 60% for the number of required sessions.

Required sessions within an AS with and without RRs

The BGP commands related to RRs include:

config router bgp

config neighbor

edit "<IPv4_IPv6_address>"

set route-reflector-client {disable | enable}

set route-reflector-client6 {disable | enable}

set route-server-client {disable | enable}

set route-server-client6 {disable | enable}

end

end

Confederations

Confederations were introduced to reduce the number of BGP advertisements on a segment of the network and reduce the size of the routing tables. Confederations essentially break up an AS into smaller units. Confederations are defined in RFC 3065 and RFC 1965.

Within a confederation, all routers communicate with each other in a full mesh arrangement. Communications between confederations is more like inter-AS communications because many of the attributes are changed as they would be for BGP communications leaving the AS, or eBGP.

Confederations are useful when merging ASs. Each AS being merged can easily become a confederation, which requires few changes. Any additional permanent changes can then be implemented over time, as required. The diagram below shows the group of ASs before merging and the corresponding confederations afterward, as part of the single AS with the addition of a new border router. It should be noted that after merging, if the border router becomes a route reflector, then each confederation only needs to communicate with one other router instead of five others.

Confederations and RRs perform similar functions: they both sub-divide large ASs for more efficient operation. They differ in that route reflector clusters can include routers that are not members of a cluster, whereas routers in a confederation must belong to that confederation. Also, confederations place their confederation numbers in the AS_PATH attribute, making it easier to trace.

NOTE: While confederations essentially create sub-ASs, all the confederations within an AS appear as a single AS to external ASs.

Confederation related BGP commands include the following:

config router bgp

set confederation-identifier <peerid_integer>

end

Network Layer Reachability Information

Network Layer Reachability Information (NLRI) is unique to BGP-4. It is sent as part of the update messages sent between BGP routers and contains information necessary to supernet, or aggregate route, information. The NLRI includes the length and prefix that, when combined, are the address of the aggregated routes referred to.

There is only one NLRI entry per BGP update message.

BGP attributes

Each route in a BGP network has a set of attributes associated with it. These attributes define the route and are modified, as required, along the route.

BGP can work well with mostly default settings, but if you're going to change settings you need to understand the roles of each attribute and how they affect those settings.

The BGP attributes include the ones listed in the following table.

Attribute

Description

AS_PATH

A list of ASs a route has passed through. For more information, see AS_PATH.

MULTI_EXIT_DESC (MED)

Which router to use to exit an AS with more than one external connection. For more information, see MULTI_EXIT_DESC.

COMMUNITY

Used to apply attributes to a group of routes. For more information, see COMMUNITY.

NEXT_HOP

Where the IP packets should be forwarded to, like a gateway in static routing. For more information, see NEXT_HOP.

ATOMIC_AGGREGATE

Used when routes have been summarized to tell downstream routers not to de-aggregate the route. For more information, see ATOMIC_AGGREGATE.

ORIGIN

Used to determine if the route is from the local AS or not. For more information, see ORIGIN.

LOCAL_PREF

Used only within an AS to select the best route to a location (like MED).

Inbound policies on FortiSwitch units can change the NEXT-HOP, LOCAL-PREF, MED, and AS-PATH attributes of an internal BGP (iBGP) route for its local route selection purposes. However, outbound policies on the device cannot affect these attributes.

AS_PATH

AS_PATH is the BGP attribute that keeps track of each AS that a route advertisement has passed through. AS_PATH is used by confederations and by exterior BGP (EBGP) to help prevent routing loops. A router knows there is a loop if it receives an AS_PATH with that router's AS in it. The diagram shows the route between Router A and Router B. The AS_PATH from A to B would read 701,702,703 for each AS that the route passes through.

As of the beginning of 2010, the industry upgraded from 2-byte to 4-byte AS_PATHs. This upgrade was due to the imminent exhaustion of 2-byte AS_PATH numbers. FortiOS supports 4-byte AS_PATHs in its BGP implementation.

AS_PATH of 701,702, 703 between routers A and B

The BGP commands related to AS_PATH include the following:

config router bgp

set bestpath-as-path-ignore {enable | disable}

end

MULTI_EXIT_DESC

BGP AS systems can have one or more routers that connect them to other ASs. For ASs with more than one connecting router, the Multi-Exit Discriminator (MED) lists which router is best to use when leaving the AS. The MED is based on attributes, such as delay. It is a recommendation only, as some networks may have different priorities.

BGP updates advertise the best path to a destination network. When a FortiSwitch unit receives a BGP update, the FortiSwitch unit examines the MED attribute of potential routes to determine the best path to a destination network before recording the path in the local FortiSwitch routing table.

FortiSwitch units have the option to treat any routes without an MED attribute as the worst possible routing choice. This can be useful because a lack of MED information is a lack of routing information, which can be suspicious as a possible hacking attempt or an attack on the network. At best, it signifies an unreliable route to select.

The BGP commands related to MED include the following:

config router bgp

set always-compare-med {enable | disable}

set bestpath-med-confed {enable | disable}

set bestpath-med-missing-as-worst {enable | disable}

set deterministic-med {enable | disable}

config neighbor

edit "<IPv4_IPv6_address>"

set attribute-unchanged [as-path] [med] [next-hop]

set attribute-unchanged6 {as-path | MED | next-hop}

end

end

end

COMMUNITY

A community is a group of routes that have the same routing policies applied to them. This saves time and resources. A community is defined by the COMMUNITY attribute of a BGP route.

A FortiSwitch unit can set the COMMUNITY attribute of a route to assign the route to predefined paths (see RFC 1997). The FortiSwitch unit can examine the COMMUNITY attribute of learned routes to perform local filtering and/or redistribution.

The BGP commands related to COMMUNITY include the following:

config router bgp

set send-community {both | disable | extended | standard}

set send-community6 {both | disable | extended | standard}

end

NEXT_HOP

The NEXT_HOP attribute says what IP address the packets should be forwarded to next. Each time the route is advertised, this value is updated. The NEXT_HOP attribute is much like a gateway in static routing.

FortiSwitch units allow you to change the advertising of the FortiSwitch unit’s IP address (instead of the neighbor’s IP address) in the NEXT_HOP information that is sent to IBGP peers. This is changed with the config neighbor, set next-hop-self command.

The BGP commands related to NEXT_HOP include the following:

config router bgp

config neighbor

edit "<IPv4_IPv6_address>"

set attribute-unchanged [as-path] [med] [next-hop]

set attribute-unchanged6 {as-path | MED | next-hop}

set next-hop-self {enable | disable}

set next-hop-self6 {disable | enable}

next

end

end

ATOMIC_AGGREGATE

The ATOMIC_AGGREGATE attribute is used when routes have been summarized. It indicates which AS and which router summarize the routes. It also tells downstream routers not to de-aggregate the route. Summarized routes are routes with similar information that have been combined, or aggregated, into one route that is easier to send in updates for. When it reaches its destination, the summarized routes are split back up into the individual routes.

The FortiSwitch unit does not specifically set this attribute in the BGP router command, but it is used in the route map command.

The CLI commands related to ATOMIC_AGGREGATE include the following:

config router route-map

edit <route_map_name>

set protocol bgp

config rule

edit <route_map_rule_id>

set set-aggregator-as <id_integer>

set set-aggregator-ip <address_ipv4>

set set-atomic-aggregate {enable | disable}

end

end

end

ORIGIN

The ORIGIN attribute records where the route came from. The options can be IBGP, EBGP, or incomplete. This information is important because internal routes (IBGP) are, by default, higher priority than external routes (EBGP). However, incomplete ORIGINs are the lowest priority of the three.

The CLI commands related to ORIGIN include the following:

config router route-map

edit <route_map_name>

set protocol bgp

config rule

edit <route_map_rule_id>

set match-origin {egp | igp | incomplete | none}

end

end

end

How BGP works

BGP is a link-state routing protocol and keeps link-state information about the status of each network link it has connected. A BGP router receives information from its peer routers that have been defined as neighbors. BGP routers listen for updates from these configured neighboring routers on TCP port 179.

A BGP router is a finite state machine with six various states for each connection. As two BGP routers discover each other and establish a connection, they go from the idle state and through the various states until they reach the established state. An error can cause the connection to drop and the state of the router to reset to either active or idle. These errors can be caused by TCP port 179 not being open, a random TCP port above port 1023 not being open, the peer address being incorrect, or the AS number being incorrect.

When BGP routers start a connection, they negotiate which (if any) optional features will be used, such as multiprotocol extensions, that can include IPv6 and VPNs.

IBGP versus EBGP

When you read about BGP, you often see EBGP or IBGP mentioned. These are both BGP routing, but BGP used in different roles. Exterior BGP (EBGP) involves packets crossing multiple autonomous systems (ASs) and interior BGP (IBGP) involves packets that stay within a single AS. For example, the AS_PATH attribute is only useful for EBGP where routes pass through multiple ASs.

These two modes are important because some features of BGP are used only for one of EBGP or IBGP. For example, confederations are used in EBGP and RRs are used only in IBGP. Also, routes learned from IBGP have priority over routes learned from EBGP.

FortiSwitch units have some commands that are specific to EBGP, including the following:

  • automatically resetting the session information to external peers if the connection goes down:set fast-external-failover {enable | disable}
  • setting an administrative distance for all routes learned from external peers (you must also configure local and internal distances if this is set):set distance-external <distance_integer>
  • enforcing EBGP multihops and their TTL (number of hops): set ebgp-enforce-multihop {enable | disable} and set ebgp-multihop-ttl <seconds_integer>

BGP path determination: Which route to use

Firstly, recall that the number of available or supported routes is not set by the configuration but depends on the available memory on the FortiSwitch unit. All learned routes and their attributes come into the BGP router in raw form. Before routes are installed in the routing table or are advertised to other routers, three levels of decisions must be made.

The three phases of BGP best path determination do not change. However, some manufacturers have added more information to the process, such as Cisco’s WEIGHT attribute, to allow an administrator to force one route’s selection over another.

There is one Adj-RIB-IN and Adj-RIB-OUT for each configured neighbor. They are updated when the FortiSwitch unit receives BGP updates or when the FortiSwitch unit sends out BGP updates.

The three phases of a BGP routing decision

Decision phase 1

At this phase, the decision is to calculate how preferred each route and its NRLI are the Adjacent Routing Information Base Incoming (Adj-RIBs-In) compared to the other routes. For internal routes (IBGP), policy information or LOCAL_PREF is used. For external peer learned routes, it is based strictly on policy. These rules set up a list of which routes are most preferred going into Phase 2.

Decision phase 2

Phase 2 involves installing the best route to each destination into the local Routing Information Base (Loc-RIB). Effectively, the Loc-RIB is the primary routing table. Each route from Phase 1 has their NEXT_HOP checked to ensure the destination is reachable. If it is reachable, the AS_PATH is checked for loops. After that, routes are installed based on the following decision process:

  • If there is only one route to a location, it is installed.
  • If there are multiple routes to the same location, use the most preferred route from Level 1.
  • If there is a tie, break the tie based on the following, in descending order of importance: shortest AS_PATH, smallest ORIGIN number, smallest MED, EBGP over IBGP, smallest metric or cost for reaching the NEXT_HOP, BGP identifier, and lowest IP address.

Note that the new routes that are installed into the Loc-RIB are in addition to any existing routes in the table. Once Phase 2 is completed, the Loc-RIB will consist of the best of both the new and older routes.

Decision phase 3

Phase 3 is route distribution or dissemination. This is the process of deciding which routes the router will advertise. If there is any route aggregation or summarizing, it happens here. Also, any route filtering from route maps happens here.

Once Phase 3 is complete, an update can be sent out to update the neighbor of new routes.

Aggregate routes and addresses

BGP-4 allows classless routing, which uses netmasks as well as IP addresses. This classless routing allows the configuration of aggregate routes by stating the address bits the aggregated addresses have in common.

The ATOMIC_AGGREGATE attribute informs routers that the route has been aggregated and should not be de-aggregated. An associated AGGREGATOR attribute include the information about the router that did the aggregating including its AS.

The BGP commands associated with aggregate routes and addresses are the following:

config router bgp

config aggregate-address

edit <aggr_addr_id>

set as-set {enable | disable}

set prefix <address_ipv4mask>

set summary-only {enable | disable}

end

end

config aggregate-address6

edit <aggr_addr_id>

set as-set {enable | disable}

set prefix6 <address_ipv6mask>

set summary-only {enable | disable}

end

end

Troubleshooting BGP

There are some features in BGP that are used to deal with problems that may arise. Typically, the problems with a BGP network that has been configured involve routes going offline frequently. This is called route flap and causes problems for the routers using that route.

Clearing routing table entries

To see if a new route is being properly added to the routing table, you can clear all or some BGP neighbor connections (sessions) using the execute router clear bgp command.

For example, if you have 10 routes in the BGP routing table and you want to clear the specific route to IP address 10.10.10.1, enter the following CLI command:

execute router clear bgp ip 10.10.10.1

To remove all routes for AS number 650001, enter the following CLI command:

execute router clear bgp as 650001

Route flap

When routers or hardware along a route go offline and back online that is called a route flap. Flapping is the term that is used if these outages continue, especially if they occur frequently.

Route flap is a problem in BGP because each time a peer or a route goes down, all the peer routers that are connected to that out-of-service router advertise the change in their routing tables. This creates a lot of administration traffic on the network and the same traffic re-occurs when that router comes back online. If the problem is something like a faulty network cable that wobbles online and offline every 10 seconds, there could easily be an overwhelming amount of routing updates sent out unnecessarily.

Another possible reason for route flap occurs with multiple FortiSwitch units in HA mode. When an HA cluster fails over to the secondary unit, other routers on the network may see the HA cluster as being offline, resulting in route flap. While this does not occur often, or more than once at a time, it can still result in an interruption in traffic that is unpleasant for network users. The easy solution for this problem is to increase the timers on the HA cluster, such as TTL timers, so they do not expire during the failover process. Also, configuring graceful restart on the HA cluster helps with a smooth failover.

The first method of dealing with route flap is to check your hardware. If a cable is loose or bad, it can easily be replaced and eliminate the problem. If an interface on the router is bad, either avoid using that interface or swap in a functioning router. If the power source is bad on a router, either replace the power supply or use a power conditioning backup power supply. These quick and easy fixes can save you from configuring more complex BGP options. However, if the route flap is from another source, configuring BGP to deal with the outages will ensure your network users uninterrupted service.

Some methods of dealing with route flap in BGP include:

Holdtime timer

The first line of defense to a flapping route is the holdtime timer. This timer reduces how frequently a route going down will cause a routing update to be broadcast.

After it is activated, the holdtime timer does not allow the FortiSwitch unit to accept any changes to that route for the duration of the timer. If the route flaps five times during the timer period, only the first outage is recognized by the FortiSwitch unit. For the duration of the other outages, there will not be changes because the FortiSwitch unit is essentially treating this router as down. If the route is still flapping after the timer expires, it'll happen all over again.

Even if the route is not flapping (for example, if it goes down, comes up, and stays back up) the timer still counts down and the route is ignored for the duration of the timer. In this situation, the route is seen as down longer than it really is but there will be only the one set of route updates. This is not a problem in normal operation because updates are not frequent.

Also, the potential for a route to be treated as down when it is really up can be viewed as a robustness feature. Typically, you do not want most of your traffic being routed over an unreliable route. So if there is route flap going on, it is best to avoid that route if you can. This is enforced by the holdtime timer.

How to configure the holdtime timer

There are three different route flapping situations that can occur: the route goes up and down frequently, the route goes down and back up once over a long period of time, or the route goes down and stays down for a long period of time. These can all be handled using the holdtime timer.

For example, your network has two routes that you want to set the timer for. One is your main route (to 10.12.101.4) that all of your Internet traffic goes through, and it cannot be down for long if it is down. The second is a low speed connection to a custom network that is used infrequently (to 10.13.101.4). The timer for the main route should be fairly short (for example, 60 seconds). The second route timer can be left at the default because it is rarely used. In your BGP configuration, this looks like the following:

config router bgp

config neighbor

edit 10.12.101.4

set holdtime-timer 60

next

edit 10.13.101.4

set holdtime-timer 180

next

end

end

Dampening

Dampening is a method that is used to limit the amount of network problems due to flapping routes. With dampening, the flapping still occurs but the peer routers pay less and less attention to that route as it flaps more often. One flap does not start dampening, but the second flap starts a timer where the router will not use that route because it is considered unstable. If the route flaps again before the timer expires, the timer continues to increase. There is a period of time called the reachability half-life, after which a route flap will be suppressed for only half the time. This half-life comes into effect when a route has been stable for a while but not long enough to clear all the dampening completely. For the flapping route to be included in the routing table again, the suppression time must expire.

If the route flapping was temporary, you can clear the flapping or dampening from the FortiSwitch unit's cache by using one of the execute router clear bgp CLI commands:

execute router clear bgp dampening {<ip_address> | <ip/netmask>}

For example, to remove route flap dampening information for the 10.10.0.0/16 subnet, enter the following CLI command:

execute router clear bgp dampening 10.10.0.0/16

The BGP commands related to route dampening are the following:

config router bgp

set dampening {enable | disable}

set dampening-max-suppress-time <minutes_integer>

set dampening-reachability-half-life <minutes_integer>

set dampening-reuse <reuse_integer>

set dampening-suppress <limit_integer>

end

BFD

Bidirectional Forwarding Detection (BFD) is a protocol that you can use to quickly locate hardware failures in the network. Routers running BFD communicate with each other and if a timer runs out on a connection then that router is declared down. BFD then communicates this information to the routing protocol and the routing information is updated. For more information about BFD, see Bidirectional forwarding detection .

Configuring BGP

Configuring BGP on the FortiSwitch unit includes the following major steps:

  1. Enter the BGP configuration mode.
  2. Set the autonomous system and router identifier.
  3. Configure a BGP neighbor.
  4. Redistribute non-BGP routes. Advertise these non-BGP routes within BGP.

1. Enter the BGP configuration mode

Enter the BGP configuration mode to access all of the BGP configuration commands:

# config router bgp

2. Set the autonomous system and router identifier

Set the autonomous system. For IBGP, the AS value needs to match the remote-as value in the neighbor router. For EBGP, the AS value differs from the remote-as value in the neighbor router. You also need to specify a fixed router identifier for the FortiSwitch unit. These two commands are mandatory.

# set as <AS number>

# set router-id <IP_address>

3. Configure the BGP neighbors

Configure the BGP neighbors.

NOTE: For IBGP, if the IP address of the BGP neighbor is a loopback address, you must use the set update-source cmd command to specify which interface address will be used as the source IP address in the outgoing BGP packet.

config neighbor

edit "<IPv4_or_IPv6 address>"

set remote-as <1-4294967295>

end

4. Redistribute non-BGP routes

Redistribute non-BGP IPv4 or IPv6 routes within BGP:

config redistribute {connected | isis | ospf | rip | static}

set status enable

set route-map <string>

end

config redistribute6 {connected | isis | ospf | rip | static}

set status {disable | enable}

set route-map <string>

end

Other BGP commands

Clearing the BGP routes

Use the following commands to clear the BGP routes:

execute router clear bgp all

execute router clear bgp ip <IPv4_or_IPv6_address>

execute router clear bgp ipv6 <IPv4_or_IPv6_address>

execute router clear bgp as <AS_number>

execute router clear bgp dampening <IP_address>

Checking the BGP configuration

The get router info bgp and get router info6 bgp commands have options to display different aspects of the BGP configuration and status.

For example:

get router info bgp neighbors

get router info bgp network

get router info6 bgp filter-list

get router info6 bgp route-map

Changing the maximum number of paths for ECMP

If you are using equal-cost multi-path (ECMP) routing with the EBGP or IBGP, the maximum number of paths is 1 by default. Use the following commands to change the default:

config router bgp

set maximum-paths-ebgp <1-64>

set maximum-paths-ibgp <1-64>

end

Sample configurations

Here is an example of a BGP routing configuration:

Configure system interfaces

Interface configuration for FortiSwitch 1:

config system interface

edit mgmt

set ip 10.105.7.9 255.255.255.0

set allowaccess ping https http ssh telnet

set type physical

next

edit internal

set type physical

next

edit vlan20-p2

set ip 192.168.2.100 255.255.255.0

set allowaccess ping https http ssh telnet

set vlanid 20

set interface internal

next

edit vlan40-p4

set ip 172.168.111.6 255.255.255.0

set allowaccess ping https http ssh telnet

set vlanid 40

set interface internal

end

config switch interface

edit "port2"

set native-vlan 20

set stp-state disabled

next

edit "port4"

set native-vlan 40

set stp-state disabled

next

edit "internal"

set allowed-vlans 1,20, 40, 4094

set stp-state disabled

next

end

Internal BGP

In this example, the two neighboring switches are in the same autonomous system.

Configuration for FortiSwitch 1:

config router bgp

set as 6500

set router-id 1.2.3.4

config neighbor

edit "172.168.111.5"

set remote-as 6500

next

end

config network

edit 1

set prefix 192.168.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

Configuration for FortiSwitch 2:

config router bgp

set as 6500

set router-id 5.6.7.8

config neighbor

edit "172.168.111.6"

set remote-as 6500

next

end

config network

edit 1

set prefix 10.50.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

External BGP

In this example, the two neighboring switches are in separate autonomous systems.

Configuration for FortiSwitch 1:

config router bgp

set as 6500

set router-id 1.2.3.4

config neighbor

edit "172.168.111.5"

set remote-as 7500

next

end

config network

edit 1

set prefix 192.168.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

Configuration for FortiSwitch 2:

config router bgp

set as 7500

set router-id 5.6.7.8

config neighbor

edit "172.168.111.6"

set remote-as 6500

next

end

config network

edit 1

set prefix 10.50.2.0 255.255.255.0

next

end

config redistribute "connected"

end

end

end

Using the following command, you can check the BGP status on the local switch:

# get router info bgp summary

To check the details about the BGP neighbors:

# get router info bgp neighbors

To check the routes learned by BGP, use the following command:

# get router info routing-table details