Traffic shaping priority queueing (PRIQ)

This section describes Priority Queueing (PRIQ), Type of Service (ToS) priority, and Quality of Service (QoS). It also explains the following:

• Why traffic shaping only occurs when traffic approaches the configured capacity on a given interface.
• Why you should configure the FortiOS to preemptively drop excess packets.
• How priority queues work on FortiOS.
• The difference between ToS-based priority and global ToS priority.
• Why you must enable traffic shaping for ALL firewall policies to get expected results.
• How firewall policy priorities and ToS policies affect each other.
• Why traffic shaper priorities only effect per port egress queueing.

Any CLI commands and GUI references in this article have been tested for both FortiOS 5.2.5 and FortiOS 5.4, and any differences between versions will be documented.

How traffic shaping really works

One of the most common misconceptions with FortiOS traffic shaping is that setting a priority will ensure that high priority traffic will download faster than low priority traffic. This perfectly reasonable expectation does not fully encapsulate what "priority" means in FortiOS, which needs to be taken into consideration. Traffic shaping will only begin to take effect when an interface with traffic shaping configured reaches its capacity. Until this threshold is reached, all traffic is treated equally. As the interface experiences high traffic levels that reach its threshold, you will begin to notice a variation in traffic flow or download speeds.

Before you begin

There are a few things you need to know about Traffic Shaping and priority queueing before you begin:

• Packets are prioritized based on their priority value.
• The priority value is based on whether you have configured Type of Service (ToS) priority or traffic shaping policy priority, or both.
• The total priority value then determines which queue the packet is placed in, out of six queue options.

• Also, remember that only per port egress queueing works.

Other considerations that affect which queue is used include:

• Whether the traffic is through traffic or originates from FortiOS.
• Whether traffic shaping will affect all traffic that is accepted by firewall policies.

Traffic shaping methods

When deciding how to configure QoS techniques, it can be helpful to know when FortiOS employs each technique in the overall traffic processing flow, and the considerations that follow.

Dropping excess packets early on

As traffic arrives (ingress) at one interface FortiOS begins to process the traffic. In later phases of network processing — such as enforcing maximum bandwidth on sessions handled by a security policy — if the current rate for the destination interface or traffic regulated by that policy is too high, FortiOS may drop the packet. Time spent on prior processing — like web filtering, decryption, or IPS — is wasted on these dropped packets.

You can prevent wasted effort on ingress by configuring FortiOS to preemptively drop excess packets when they are received at the source interface, before most other traffic processing is performed:

config system interface

edit <interface_name>

set inbandwidth <rate_int>

set outbandwidth <rate_int>

end

Where <rate_int> is set to the bandwidth limit in Kb/s, excess packets will be dropped. If the inbandwidth <rate_int> is set to 0, then the rate is not limited.

As with ingress, if you set the rate to 0 (zero) you are setting the rate to unlimited. Rate limiting traffic accepted by the interface enables you to restrict incoming traffic to rates that, while no longer the full capacity of the interface, at the traffic shaping point in the processing are more likely to result in acceptable rates of outgoing traffic per destination interface or all security policies. This conserves processing resources for those packets likely to be viable (to the point of egress).

The following diagram shows how excess packets going from LAN to WAN 1 can be intercepted and dropped at the source interface.

How priority queuing works

After packet acceptance, FortiOS classifies traffic and may apply traffic policing at additional points during processing. FortIOS may also apply QoS techniques, such as prioritization and traffic shaping. Traffic shaping consists of a mixture of traffic policing to enforce bandwidth limits, and priority queue adjustment to assist packets in achieving the guaranteed rate.

If you have configured prioritization, FortiOS prioritizes egressing packets by distributing them among FIFO (first in, first out) queues associated with each possible priority number. Each physical interface has six priority queues. Virtual interfaces use the priority queues of the physical interface to which they are bound.

Each physical interface's six queues are queue 0 to queue 5, where queue 0 is the highest priority queue. However, you may observe that your traffic uses only a subset of those six queues. For example, some traffic may always use a certain queue number. Queuing may also vary by the packet rate or mixture of services. Some queue numbers may only be used by through traffic for which you have configured traffic shaping in the security policy that applies to that traffic session.

• Administrative access traffic will always use queue 0.
• Traffic matching security policies without traffic shaping may use queue 0, queue 1, or queue 2. The queue is selected based on the priority value you have configured for packets with that ToS (Type of Service) bit value, if you have configured ToS-based priorities.
• Traffic matching security policies with traffic shaping enabled in the policy may use any queue. The queue is selected based on whether the packet rate is currently below the guaranteed bandwidth (queue 0), or above the guaranteed bandwidth. Packets at rates greater than the maximum bandwidth limit are dropped.
• If the global tos-based-priority is low (3) and the priority in a traffic-shaper is medium (2), when a packet flows through a policy that refers to the shaper, the packet will be assigned the priority defined by the shaper. In this case, medium (2).

Types of priority

Prioritization and traffic shaping behavior vary based on the configuration, service type, traffic volume, and whether the traffic is through traffic or originates from FortiOS.

Packets can be assigned a priority in one of three ways:

• On ingress - for packets flowing through the firewall.
• Upon generation - for packets generated by the firewall (including packets generated due to AV proxying).
• On passing through a firewall policy - for packets that matches a traffic shaping policy.

Ingress priority and priority for generated packets is controlled by two different CLI settings:

config system global

set traffic-priority-level {high | medium | low}

end

config system tos-based-priority

edit 1

set tos [0-15]

set priority (high | medium | low)

end

Type of Service (ToS) priority

Type of Service is an 8-bit field in the IP header that allows you to determine how an IP datagram should be delivered, using the following criteria: Delay, Throughput, Priority, Reliability, and Cost. The criteria help gateways pick the best way to route datagrams.

A router maintains a ToS value for each route in its routing table. The lowest priority ToS is 0, and the highest is 7 (when bits 3, 4, and 5 are all set to 1). There are four other bits that are seldom used or reserved that are not included here.

Together these bits are the tos variable of the tos-based-priority command. The router tries to match the ToS of the datagram to the ToS on one of the available routes to the destination. If there is no match, then the datagram is sent over a zero ToS route. Using increased quality may increase the cost of delivery, because better performance may consume limited network resources.

Each bit represents the priority as per RFC 1349:

• 1000 - minimize delay
• 0100 - maximize throughput
• 0010 - maximize reliability
• 0001 - minimize monetary cost

The tos value is set in the CLI using the following command:

config system tos-based-priority

edit <sequence_number>

set tos [0-15]

set priority [high | medium | low]

end

Where tos is the value of the type of service bit in the IP datagram header with a value between 0 and 15, and priority is the priority of this type of service.

ToS priority	Setting
High	1
Medium	2
Low	3

These priority levels conform to the firewall traffic shaping priorities, as defined in RFC 1349.

Traffic shaper priority

You set priorities for individual traffic shapers to high, medium, or low.

Traffic shaper priority	Setting
High	1
Medium	2
Low	3

When you add a traffic shaper to a traffic shaping policy, the shaper takes effect after any DoS detection policies and before any routing or packet scanning occurs.

The shaper you select for a traffic shaping policy (shared shaper) will affect the traffic in the direction defined in the policy. For example, if the source port is lan and the destination is wan1, the shaping affects the flow in this direction only — affecting the upload speed of the outbound traffic.

By selecting Shared Traffic Shaper Reverse Direction, you can define the traffic shaper for the policy in the opposite direction to affect the download speed of the inbound traffic. In this example, from wan1 to lan.

config firewall shaping-policy

edit 0

set traffic-shaper <shaper_name>

set per-ip-shaper <shaper_name>

set traffic-shaper-reverse <shaper_name>

end

To have proper QoS, the traffic shaping policy you create between your incoming interface and outgoing interface should include two interfaces. For example, a LAN to WAN1 policy.

Traffic Shaping Policies allow you to apply traffic shaping globally to any traffic matching your criteria. The criteria must specify a source, a destination, a service, and the outgoing interface.

How do these priorities affect each other?

The global or ingress ToS-based priority value is combined with the shaper priority value:

Global priority (0, 1, 2) + shaper priority (1, 2, 3) = total priority (queue number).

Let's take a look at some examples:

• Assuming a default ingress priority of low (2) and a firewall policy priority of low (3), then the resulting priority is 5.
• If the packet flowing through results in a rate that is less than the guaranteed bandwidth, then the priority is set to 0 regardless of the priority in the firewall policy.
• If the packet flowing through results in a rate that's above the maximum bandwidth, then the packet is dropped.
• If the packet flowing through results in a rate that is between the guaranteed and the maximum bandwidth, then the packet priority is increased by the priority from the shaper. Therefore, assuming a default ingress priority of high (0) and a shaper policy of high (1), then the resulting priority is 1.
• When a packet is sent to the egress device, it is attached to a queue based on the packet priority. For example, priority 0 is attached to queue 1, and so on. If the queue is full, then the packet is dropped.

Shaper priority only affects per interface egress queueing. Thus, if there are two streams of traffic — with one egressing over port1 and one egressing over port2 — then the priority has no effect whatsoever. Both streams will continue to run at full speed.

Traffic passing through FortiOS

The method FortiOS uses to determine the priority queue for traffic passing through FortiOS depends on whether you have enabled Traffic Shaping. Packets may or may not use a priority queue directly or indirectly derived from the type of service (ToS) bit — sometimes used instead with differentiated services — in the packet's IP header.

If a traffic shaping does not match a traffic flow, FortiOS neither limits nor guarantees bandwidth. Traffic shaping for that session uses the priority queue determined by matching the ToS bit in its header with your configured values:

config system global

set traffic-priority-level {high | medium | low}

end

or, if you have configured a priority specifically for that TOS bit value:

config system tos-based-priority

edit 0

set tos [0-15]

set priority {high | medium | low}

end

Where the value of tos is the value of the ToS bit in the packet's IP header, and high has a value of 0 and low is 2. Priority values configured in the second location will override the global ToS-based priority. In other words, packet priority = ToS-based priority.

For example, you might specify that packets with a ToS bit value of 2 should use queue 0, the highest priority queue:

config system tos-based-priority

edit 15

set tos 2

set priority high

end

If Traffic Shaping is enabled in a traffic shaping policy using shared traffic shapers, FortiOS may instead or also subject packets to traffic policing or priority queue increases to meet bandwidth guarantees configured in the shaper:

config firewall shaper traffic-shaper

edit <shaper_name>

set priority {high | medium | low }

set maximum bandwidth <rate>

set guaranteed-bandwidth <rate>

end

Where high has a priority value of 1 and low is 3, and <rate> is the bandwidth limit in kilobits per second.

Traffic queueing as the packet rate increases.

• If the current packet rate is less than the guaranteed bandwidth, packets use priority queue 0. In other words, packet priority = 0.
• If the current packet rate is greater than the guaranteed bandwidth, but less than maximum bandwidth, FortiOS assigns a priority queue by adding the numerical value of the shaper priority, where high has a priority value of 0 and low is 2. Because the two values are added, depending on the configured ToS-based priorities, packets in this category could use queues from queue 1 to queue 5. In other words, packet priority = ToS-based priority + security policy-based priority. For example, if you have enabled Traffic Shaping in the security policy, and the security policy's Traffic Priority is Low (value 3), and the priority normally applied to packets with that ToS bit is medium (value 1), then packets have a total packet priority of 4, and use priority queue 4.
• If the current packet rate exceeds the maximum bandwidth, excess packets are dropped.

Traffic originating from FortiOS

FortiOS does not apply traffic shaping to administrative access traffic to FortiOS through HTTPS or SSH, or IPsec tunnel negotiations. These types of traffic use the highest priority queue, queue 0. In other words, packet priority = 0.

Exceptions to this rule include traffic types with connections that are related to a session governed by a security policy. For example, if you have enabled proxy-based AntiVirus scanning, traffic from the sender technically terminates at the proxy that scans that traffic type; FortiOS initiates a second connection that transmits scanned content to its destination. Because the second connection's traffic is technically originating from the proxy, and therefore FortiOS, it uses the highest priority queue, queue 0. However, this connection is logically associated with through traffic, and is therefore subject to possible bandwidth enforcement and guarantees in its governing security policy. In this way, it behaves partly like other through traffic.

Egress queueing

Shaper priority only affects per port egress queueing, so if you have two streams of traffic — like one egressing over port1 and one egressing over port2 — then priority has no effect whatsoever. Both streams will continue to run at full speed.

To make any difference to the order in which packets egress the interface, there must be packets of a lower priority queued on the egress interface. This usually happens when there is an imbalance between the packet rates on the interfaces.

For example, if the LAN is 1Gb, but the WAN is only 100MB. In this scenario the priority of the traffic egressing the WAN is very important, but the traffic egressing the LAN is rendered irrelevant (as it would take 10 WAN links to drive traffic at a high enough rate to cause queuing interference on the LAN interface).

This was tested by performing a debug on the kernel to determine when priority would take effect. In this case, by counting how many times the egress interface had more than one packet in the queue. Two simultaneous 500MB downloads via HTTP were performed, with one policy set to a high priority and one set to a low priority. Results showed that there was more than one packet in the egress queue only 23 times. With over 600,000 packets egressing over that interface, altering the priority of 23 does not make a practical difference to the relative speed of downloads.