Overview

QoS is the capability to adjust quality aspects of your overall network traffic, including techniques such as priority-based queuing and traffic policing. Because bandwidth is finite and some types of traffic are slow, jitter or packet loss sensitive, bandwidth intensive, or operation critical, QoS can be a useful tool for optimizing the performance of various applications in your network. QoS is especially important in managing voice and streaming multimedia traffic because these types of traffic can rapidly consume bandwidth and are sensitive to latency. You can implement QoS on FortiGate devices using the following techniques:

Technique	Description
Traffic policing	The FortiGate drops packets that don't conform to the configured bandwidth limitations. Note that excessive traffic policing can degrade network performance rather than improve it.
Traffic shaping	The FortiGate ensures that traffic consumes bandwidth at least at the guaranteed rate by assigning a greater priority queue to the traffic if the guaranteed rate isn't being met. The FortiGate ensures that traffic doesn't consume more bandwidth than the configured maximum bandwidth. Traffic that exceeds the maximum rate is subject to traffic policing.
Queuing	Transmits packets in the order of their assigned priority queue for that physical interface. All traffic in a higher priority traffic queue must be completely transmitted before traffic in lower priority queues is transmitted.

When you're determining how to configure QoS, it's helpful to know when a FortiGate device employs each technique in the overall flow of traffic processing and the considerations for each technique. After the FortiGate accepts packets, it classifies the traffic and may apply traffic policing at additional points during traffic processing. The FortiGate may also apply QoS techniques, such as prioritization and traffic shaping. Traffic shaping consists of both traffic policing to enforce bandwidth limits and adjusting priority queues to help packets achieve the guaranteed rate.

If you configure prioritization, the FortiGate prioritizes egress packets by distributing them among FIFO (first in, first out) queues that are associated with each priority number. Each physical interface on the FortiGate has six priority queues, queue 0 to queue 5, where queue 0 is the highest priority queue. Virtual interfaces don't have their own queues, and instead use the priority queues of the physical interface to which they are bound.

Network traffic may use only a subset of the six queues. Certain types of traffic may always use a specific queue number. Queuing may vary by packet rate or mix of services. Some queue numbers may be used only by through traffic for which you have configured traffic shaping in the security policy that applies to that traffic session. Administrative access traffic uses queue 0. If you configured ToS-based priorities, traffic that matches security policies without traffic shaping may use queue 0, 1, or 2, depending on the priority value that you configured for packets with that Type of Service (ToS) bit value. Traffic that matches security policies with traffic shaping can use any of the queues, depending on whether the packet rate is above or below the guaranteed bandwidth (below uses queue 0). For example, if the global tos-based-priority is low (3), the priority in a traffic shaper is medium (2) and a packet flows though a policy that refers to the traffic shaper. The packet is assigned the priority that's defined by the traffic shaper, which is medium (2).

Security policies don't apply to administrative access to the FortiGate through HTTPS or SSH, or IPsec tunnel negotiations, and therefore the FortiGate doesn't apply traffic shaping to this type of traffic. This traffic uses the highest priority queue, which is queue 0 (packet priority = 0). Exceptions to this rule include traffic types that are connections related to a session governed by a security policy. For example, if you enable FortiGuard antivirus scanning, traffic from the sender terminates at the FortiGate proxy that scans that type of traffic. The FortiGate initiates a second connection that transmits scanned content to its destination. Because the second connection’s traffic originates from the FortiGate proxy, and therefore the FortiGate itself, it uses the highest priority queue (queue 0). However, this connection is logically associated with through traffic, and is therefore subject to possible bandwidth enforcement and guarantees in its governing security policy. In this way, it behaves partly like other through traffic.

The results of traffic prioritization and traffic shaping vary according to the configuration, service types and traffic volumes, and whether the traffic the traffic originates from or terminates at the FortiGate or is through traffic. Through traffic refers to traffic that passes through the FortiGate. The method that a FortiGate uses to determine the priority queue for through traffic depends on whether traffic shaping is enabled or not. Packets can use a priority queue directly or indirectly derived from the Type of Service (ToS) bit in the packet's IP header (sometimes used instead with differentiated services).

If traffic shaping isn't applied to a security policy, the FortiGate doesn't limit or guarantee bandwidth and traffic for that session uses the priority queue determined directly by matching the ToS bit in its header with the ToS priority configuration on the FortiGate. If traffic shaping is applied to a security policy using a shared traffic shaper, the FortiGate may subject packets to traffic policing or priority queue increases in an effort to meet bandwidth guarantees configured in the traffic shaper.

If the current packet rate is less than guaranteed bandwidth, packets use priority queue 0 (packet priority = 0). If the current packet rate is greater than guaranteed bandwidth, but less than the maximum bandwidth, the FortiGate assigns a priority queue by adding the numerical value of the security policy-based priority, where the value of High is 1, and Low is 3, with the numerical value of the ToS-based priority, where high has a priority value of 0 and low is 2. Because the two values are added, depending on the configured ToS-based priorities, packets in this category could use queues from queue 1 to queue 5. In other words, packet priority = ToS-based priority + security policy-based priority. If you enable traffic shaping in the security policy, and the security policy traffic priority is low (value 3), and the priority that's normally applied to packets with that ToS bit is medium (value 1), the packets have a total packet priority of 4, and use priority queue 4.

Packet rates

The formula for packet rates specified for Maximum Bandwidth or Guaranteed Bandwidth are:

rate = amount / time

where rate is in Kbps

Burst size can't exceed the configured maximum bandwidth. The FortiGate drops packets that exceed the configured maximum bandwidth. Packets deduct from the amount of bandwidth available to subsequent packets and available bandwidth regenerates at a fixed rate. As a result, the bandwidth that's available to a packet may be less than the configured rate, down to a minimum of 0 Kbps.

Alternatively, rate calculation and behavior can be described using the token bucket metaphor. A traffic flow has an associated bucket, which represents burst size bounds and is the size of your configured bandwidth limit. The bucket receives tokens, which represent available bandwidth at the fixed configured rate. As time passes, tokens are added to the bucket, up to the capacity of the bucket and excess tokens are discarded. When a packet arrives on the FortiGate, the packet must deduct bandwidth tokens from the bucket equal to its packet size in order to leave the FortiGate. Packets can't leave the FortiGate if there aren't enough tokens to pay for it to leave the FortiGate, and these packets are dropped.

Bursts aren't redistributed over a longer interval, so bursts are propagated rather than smoothed. However, peak size is limited. The maximum burst size is the capacity of the bucket, which is the configured bandwidth limit. The actual size varies depending on the current number of tokens in the bucket, which may be less than the capacity of the bucket, due to deductions made by previous packets and the fixed rate at which tokens accumulate. A bucket that's depleted refills at the rate of the configured bandwidth limit. Bursts can't borrow tokens from other time intervals.

By limiting traffic peaks and token regeneration in this way, the available bandwidth may be less than the capacity of the bucket, but the limit of the total amount per time interval is ensured. Total bandwidth use during each interval of one second is, at most, the integral of your configured rate.

You may observe that external clients, such as FTP or BitTorrent clients, initially report rates between the maximum bandwidth and twice the amount of the maximum bandwidth, depending on the size of their initial burst. For example, when a connection is initiated following a period of no network activity. The apparent discrepancy in rates is caused by a difference in perspective when delimiting time intervals. A burst from the client may initially consume all tokens in the bucket, and before the end of one second, as the bucket regenerates, is allowed to consume almost another bucket’s worth of bandwidth. From the perspective of the client, this equals one time interval. However, from the perspective of the FortiGate, the bucket can't accumulate tokens when it's full. Therefore, the time interval for token regeneration begins after the initial burst and doesn't contain the burst. These different points of reference result in an initial discrepancy that's equal to the size of the burst. The client’s rate contains it, but the FortiGate device’s rate doesn't. However, if the connection is sustained to its limit and time progresses over an increasing number of intervals, this discrepancy decreases in importance relative to the bandwidth total and the client’s reported rate will eventually approach that of the configured rate limit for the FortiGate.

For example, the maximum bandwidth might be 50 Kbps, there has been no network activity for one or more seconds, and the bucket is full. A burst from an FTP client immediately consumes 50 kilobits. Because the bucket completely regenerates over one second, by the time another second elapses from the initial burst, traffic can consume another 49.999 kilobits, for a total of 99.999 kilobits between the two points in time. From the vantage point of an external FTP client regulated by this bandwidth limit, it therefore initially appears that the bandwidth limit is 99.999 Kbps. This is almost twice the configured limit of 50 Kbps. However, bucket capacity regenerates only at your configured rate of 50 Kbps and the connection can only consume a maximum of 50 kilobits during each subsequent second. The result is that as bandwidth consumption is averaged over an increasing number of time intervals, each of which are limited to 50 Kbps, the effect of the first interval’s doubled bandwidth size diminishes proportionately, and the client’s reported rate eventually approaches your configured rate limit. The following table shows the effects of a 50 Kbps limit on client reported rates:

Total size transferred (kilobits)	Time (seconds)	Rate reported by client (Kbps)
99.999 (50 + 49.999)	1	99.999
149.999	2	74.999
199.999	3	66.666
249.999	4	62.499
299.999	5	59.998
349.999	6	58.333
...	...	...

Guaranteed bandwidth can also be described using a token bucket metaphor. However, because this feature attempts to achieve or exceed a rate rather than limit it, the FortiGate doesn't discard non-conforming packets, as it does for maximum bandwidth. Instead, when the flow doesn't achieve the rate, the FortiGate increases the packet priority queue, in an effort to increase the rate.

Guaranteed and maximum bandwidth rates apply to the bidirectional total for all sessions controlled by the security policy. For example, an FTP connection may entail two separate connections for the data and control portion of the session. Some packets may be reply traffic rather than initiating traffic. All packets for both connections are counted when calculating the packet rate for comparison with the guaranteed and maximum bandwidth rate.

Determining your QoS requirements

Before you implement QoS, you should identify the types of traffic that:

• Are important to your organization
• Use high amounts of bandwidth
• Are sensitive to latency or packet loss

Discovering the needs and relative importance of each traffic type on your network helps you to design an appropriate overall approach, including how you configure each available QoS component technique. Some organizations discover that they only need to configure bandwidth limits for some services. Other organizations determine that they need to fully configure interface and security policy bandwidth limits for all services, and prioritize queuing of critical services relative to traffic rate.

For example, your organization may want to guarantee sufficient bandwidth for revenue producing e-commerce traffic. You may need to ensure that transactions are completed and your clients don't experience service delays. At the same time, you may need to ensure low latency for voice over IP (VoIP) traffic that sales and customer support teams use, while traffic latency and bursts may be less critical to the success of other network applications, such as long term, resumable file transfers.

Accelerated interfaces (NPx network processors and CE) affect traffic shaping. For more information, see FortiOS Hardware Acceleration.

Configuring QoS using priority values defined in security policies

Configurations implementing QoS using the priority values defined in the security policies are capable of applying bandwidth limits and guarantees.

In addition to configuring traffic shaping, you may also choose to limit the bandwidth accepted by each interface. This can be useful in scenarios where the bandwidth received on source interfaces frequently exceeds the maximum bandwidth limit defined in the security policy. Rather than waste processing power on packets that will get dropped later in the process, you may choose to preemptively police the traffic.

If you decide to implement QoS using security policies rather than ToS bit, the FortiGate applies QoS to all packets controlled by the policy. This type of control is less granular than prioritization by ToS bit, but has the benefits of correlating quality of service to a security policy. This correlation enables you to distribute traffic over up to four of the possible 6 priority queues (queue 0 to queue 3), doesn't require other devices in your network to set or respect the ToS bit, and enables you to configure bandwidth limits and guarantees.

Configuring QoS using priority values defined in ToS bit values

Configurations implementing QoS using the priority values defined in either global or specific ToS bit values are not capable of applying bandwidth limits and guarantees, but are capable of prioritizing traffic at per-packet levels, rather than uniformly to all services matched by the security policy.

In addition to configuring traffic prioritization, you may also choose to limit bandwidth that's received by each interface. This can sometimes be useful in scenarios where you want to limit traffic levels, but don't want to configure traffic shaping within a security policy. This has the benefit of policing traffic at a point before the FortiGate performs most processing.

Note that if you implement QoS using ToS octet rather than security policies, the FortiGate applies QoS on a packet-by-packet basis, and priorities may be different for packets and services controlled by the same security policy. This is more granular control than prioritization by security policies, but has the drawbacks that quality of service may not be uniform for multiple services controlled by the same security policy, packets only use up to three of the six possible queues (queue 0 to queue 2), and bandwidth can't be guaranteed. Other devices in your network must also be able to set or preserve ToS bits.

VLAN, VDOM, and virtual interfaces

Policy-based traffic shaping doesn't use queues directly. It shapes the traffic and if the packet is allowed by the security policy, a priority is assigned. That priority controls what queue the packet is put in on egress. VLANs, VDOMs, aggregate ports, and other virtual devices don't have queues and, as such, traffic is sent directly to the underlying physical device where it's queued and affected by the physical ports. This is also the case with IPsec connections.

Best practices

The following list includes recommendations and considerations when you're configuring QoS in your network:

• For maximum bandwidth limits, ensure that bandwidth limits at the source interface and security policy aren't too low. This can cause the FortiGate to discard an excessive number of packets.
• For prioritization, consider the ratios of how packets are distributed between the available queues, and which queues are used by which types of services. Assigning most packets to the same priority queue can reduce the effects of configuring prioritization. Assigning a lot of high bandwidth services to high priority queues may take too much bandwidth away from lower priority queues and cause increased or indefinite latency. For example, you may want to prioritize a latency-sensitive service, such as SIP, over a bandwidth-intensive service, such as FTP. Also consider that bandwidth guarantees can affect queue distribution, and assign packets to queue 0 instead of their regular queue in high-volume situations.
• You may or may not want to guarantee bandwidth because it causes the FortiGate to assign packets to queue 0 if the guaranteed packet rate isn't being met. Comparing queuing behavior for low and high bandwidth situations, this means that the effect of prioritization becomes visible only as traffic volumes rise and exceed their guarantees. Because of this, you might want only some services to use bandwidth guarantees so that you can avoid the possibility that all traffic uses the same queue in high-volume situations, which negates the effects of configuring prioritization.
• For prioritization, configure prioritization for all through traffic. Configuring prioritization by either ToS-based priority or security policy priority, but not both, simplifies analysis and troubleshooting. Traffic subject to both security policy and ToS-based priorities use a combined priority from both parts of the configuration, while traffic subject to only one of the prioritization methods will use only that priority. If you configure both methods, or if you configure either method for only a subset of traffic, packets that apply to the combined configuration may receive a lower priority queue than packets that apply to only one of the priority methods, as well as packets that don't apply to the configured prioritization. For example, if both the ToS-based priority and security policy priority dictate that a packet should receive a medium priority, in the absence of bandwidth guarantees, a packet uses queue 3. If only ToS-based priority is configured, the packet uses queue 1. If only security policy-based priority is configured, the packet uses queue 2. If no prioritization is configured, the packet uses queue 0.
• Packets that are discarded by traffic shapers impact flow-control mechanisms, such as TCP. For more accurate testing results, you should use the UDP protocol.
• Traffic shaping accuracy is optimal for security policies without a protection profile where no FortiGate content inspection is processed.
• Don't oversubscribe an outbandwith throughput. For example, sum [guaranteed bandwidth] < outbandwith. For accuracy in bandwidth calculation, you must set the outbandwidth parameter on interfaces.
• The FortiGate doesn't prioritize traffic based on the DSCP marking that's configured in the security policy. However, ToS-based prioritizing can be done for ingress traffic.
• While it's possible to configure QoS using a combination of security policies and ToS ­based priorities, and to distribute traffic over the six possible queues for each physical interface, the results of those configurations can be more difficult to analyze because of their complexity. In those cases, prioritization behavior can vary by several factors, including traffic volume, type of service (ToS) or differentiated services (DiffServ) markings, and correlation of session to a security policy.