Understanding FortiDDoS rate limiting thresholds

This section includes the following information:

• Granular monitoring and rate limiting
• Source tracking table
• Destination tracking table
• Continuous learning and adaptive thresholds
• Hierarchical nature of protocols and implication on thresholds

Granular monitoring and rate limiting

Increasingly, instead of simple bandwidth attacks, attackers try to avoid detection by creating attacks that mimic the behavior of a large number of clients. Evading an NBA system is easy if attackers do coarse-grained rate-based control. Because the content of the malicious requests does not differ from that of legitimate ones, the resulting attacks are hard to defend against using standard techniques.

In contrast, FortiDDoS-F uses a combination of Layer 3, 4, and 7 counters and continuously adapts expected inbound and outbound rates for each of these traffic parameters.

Granular analytics also enable targeted mitigation responses. For example, if a few TCP connections are exceeding bandwidth, the system blocks those connections rather than all connections. If a single destination is under attack, FortiDDoS-F drops packets to that destination while others continue. During fragmented flood attacks, all non-fragmented packets continue as usual. During a port flood to a non-service port, the packets to other ports continue.

Granularity helps to increase the goodput (the throughput of useful data) of the system.

The table below lists the counters that FortiDDoS uses to detect subtle changes in the behavior of network traffic.

FortiDDoS-F counters

Type	200F	1500F	VM-04	VM-08	VM-16
Service Protection Policies (SPPs)	8	16	4	8	16
Protection Subnets per SPP	512	1024	512	512	512
ACLs (system unless noted)
Global ACLs IPv4	1024
Global ACLs IPv6	1024
ACLs per SPP (IPv4 or IPv6)	1024
Blocklist IPv4 upload list	1 million
Blocklist Domains	1024
Domain Blocklist upload list	1 million
Layer 3
Protocol flood (Protocol pps rate, per SPP, per direction)	1 counter/Threshold for each of 256 protocols
Fragment flood (Fragment pps rate. Per SPP per direction)	3 counters/Thresholds for TCP/UDP/Other Fragments
IP source flood & source tracking (Source IPs per system)	1 million	4 million	1 million	1 million	2 million
IP destination flood (Destination Ips per system)	1 million	4 million	1 million	1 million	2 million
Layer 4
TCP port flood (Port data pps Thresholds per SPP per direction)	65,535 (Port 0 is anomaly)	65,535 (Port 0 is anomaly)	65,535 (Port 0 is anomaly). Ports above 1023 are all graphed into port 1024
UDP port flood (Port data pps rate Thresholds per SPP per direction)	65,535 (Port 0 is anomaly)	65,535 (Port 0 is anomaly)	65,535 (Port 0 is anomaly). Ports above 10239 are all graphed into port 10240
ICMP type/code flood (Type/Code pps rate Thresholds per SPP per direction)	65,536	65,536	65,536 Type/Code indexes. Indexes above 10239 are all graphed into index 10240
TCP session table (sessions per system)	4 million	16 million	4 million	4 million	8 million
Legitimate IP table ( IP addresses per system) (used to store legitimate Sources during SYN or DNS Query Floods)	1 million	4 million	1 million	1 million	2 million
SYN flood (SYN Rate pps per SPP per direction)	6.5 million	14.7 million	2 million	2 million	2 million
SYN/source (Sources tracked per system)	1 million	4 million	1 million	1 million	2 million
SYN/destination (Destinations tracked per system)	1 million	4 million	1 million	1 million	2 million
Concurrent connections/source (Sources tracked per system)	1 million	4 million	1 million	1 million	2 million
Layer 7
HTTP method (per Method pps rate per SPP per direction)	8 Method counters/Thresholds(GET, POST, HEAD, PUT, DELETE, CONNECT, OPTIONS, TRACE)
HTTP Method/Source (1 Threshold per SPP per direction) Total Sources tracked per system	1 million	4 million	1 million	1 million	2 million
URLs (1 Threshold per SPP per direction) URLs tracked per SPP per direction	Top 64,000	Top 64,000	Top 64,000	Top 64,000	Top 64,000
Hosts (1 Threshold per SPP per direction) Hosts tracked per SPP per direction	Top 512	Top 512	Top 512	Top 512	Top 512
Referer (1 Threshold per SPP per direction) Referers tracked per SPP per direction	Top 512	Top 512	Top 512	Top 512	Top 512
Cookie (1 Threshold per SPP per direction) Cookies tracked per SPP per direction	Top 512	Top 512	Top 512	Top 512	Top 512
User-Agent (1 Threshold per SPP per direction) User-Agents tracked per SPP per direction	Top 512	Top 512	Top 512	Top 512	Top 512
DQRM (DNS Query/Responses tracked per system)	2 million	8 million	2 million	2 million	4 million
DNS LQ (cached Rcode-0 FQDNs per system)	131,000	524,000	131,000	131,000	262,000
DNS TTL (Cached TTLs for Rcode-0 FQDNs per system)	2 million	8 million	2 million	2 million	4 million
DNS Cache (Cached Rcode-0 Responses per system)	65,500	262,000	65,536	65,536	131,000
DNS query (QPS rate per SPP per direction)	2 million	8 million	2 million	2 million	4 million
DNS query rate/Source (Sources tracked per system)	1 million	4 million	1 million	1 million	2 million
DNS suspicious activity/source (Sources tracked per system)	1 million	4 million	1 million	1 million	2 million
DNS question count (Qcount sum per second per SPP per direction)	2 counters (UDP, TCP)
DNS MX count (Qps rate per SPP per direction)	2 counters (UDP, TCP)
DNS All count (Qps rate per SPP per direction)	2 counters (UDP, TCP)
DNS zone transfer count (Qps rate per SPP per direction)	1 counter
DNS fragment count (DNS Fragments per second rate per SPP per direction)	2 counters (UDP, TCP)
DNS Response Code count (Responses per second rate per SPP per direction)	16 counters, one for each Rcode
NRM (NTP Requests/Responses tracked per system)	4 million	16 million	4 million	4 million	8 million
NTP Requests (Requests per second, per SPP per direction)	1 counter
NTP Response (Responses per second, per SPP per direction)	1 counter
NTP Broadcast (Broadcast pps , per SPP per direction)	1 counter
NTP Response pre Destination (Responses per second) (Destinations Tracked)	1 million	4 million	1 million	1 million	2 million

Source tracking table

FortiDDoS maintains massive connection tables of 4-16 million entries and still performs with very low latency. FortiDDoS does not use its connection tables for non-TCP traffic and thus is immune to ICMP, Fragment and UDP table-filling attacks. SYN packets are validated under flood without populating the connection table (non-proxy).

The source tracking table enables FortiDDoS to correlate sources with attack events and apply a more stringent blocking period to the sources that exceeded maximum rate limits. The source tracking table also enables the special “per-source” thresholds described in the table below.

All per-source thresholds are part of FortiDDoS "Scalar" thresholds and as such all have a machine-learned adaptive Thresholds used in conjunction with the system Recommended Thresholds created from the learned traffic. This "Estimated Threshold" compensates for traffic seasonality over short periods of time (from a few minutes to 6 weeks).

Per-source thresholds

Counter	Description
most-active-source	This counter establishes a maximum packet rate for any IP packet from a single source. A rate that exceeds the adjusted baseline is anomalous and treated as a Source Flood attack event. In conjunction with the Source Multiplier, the most-active-source threshold is useful in tracking and blocking non-spoofed sources that are participating in an attack. See the figure: System attack response timeline. How is the threshold determined? When it establishes baseline traffic statistics, FortiDDoS records the highest packet rate from a single source during the observation period. In a one hour observation period, FortiDDoS collects a data point for twelve five minute windows. The data point is the highest rate observed in any one second during the five minute window. If the packet rate data points for most-active-source are 1000, 2000, 1000, 2000, 1000, 2000, 1000, 2000, 3000, 2000, 1000, and 2000, the generated statistic is the highest one: 3000.
syn-per-src	This counter establishes a maximum packet rate for SYN packets from a single source. A rate that exceeds the adjusted baseline is anomalous and treated as a SYN Flood From Source attack event.
concurrent-connections-per-source	This counter establishes a maximum packet rate for concurrent connections from a single source. A count that exceeds the adjusted baseline is anomalous and treated as an Excessive Concurrent Connections Per Source attack event.
dns-query-per-src	This counter establishes the maximum rate of DNS queries from a single source. A count that exceeds the adjusted baseline is anomalous and treated as DNS Query Flood From Source attack event.
dns-packet-track-per-src	This counter is based on heuristics to detect suspicious actions from sources. The source tracking counter is incremented when a query is not found in the DQRM, when there are fragmented packets in the query or response, and when the response has an RCODE other than 0.
methods-per-source	Drops due to method per source threshold.

Destination tracking table

FortiDDoS destination table supports 1-4 million entries depending on the model.

This table tracks the packet rate for every destination and is used for “per-destination” thresholds.

The destination tracking table enables FortiDDoS to prevent destination flood attacks and slow connectionattacks that are targeted at individual destinations. The “per-destination” thresholds enable it to do so without affecting the rates for other destinations in the SPP.

All per-destination thresholds are part of FortiDDoS "Scalar" thresholds and as such all have a machine-learned adaptive Thresholds used in conjunction with the system Recommended Thresholds created from the learned traffic. This "Estimated Threshold" compensates for traffic seasonality over short periods of time (from a few minutes to 6 weeks).

The table below describes the per-destination thresholds.

Per-destination thresholds

Counter	Description
most-active-destination	This counter establishes a maximum packet rate to any one destination. A rate that exceeds the adjusted baseline is anomalous and treated as a Destination Flood attack event. How is the threshold determined? When it establishes baseline traffic statistics, FortiDDoS records the highest packet rate to any single destination during the observation period. In a one hour observation period, FortiDDoS collects a data point for twelve five minute windows. The data point is the highest rate observed in any one second during the five minute window. If the packet rate data points for most-active-destination are 100000, 200000, 100000, 200000, 100000, 200000, 100000, 200000, 300000, 200000, 100000, and 2000, the generated statistic is the highest one: 300000.
syn-per-dst	This counter establishes a maximum packet rate for particular TCP packets to a single destination. A rate that exceeds the adjusted baseline is anomalous and treated as a Excessive TCP Packets Per Destination flood attack event. When the syn-per-dst limits are exceeded for a particular destination, the SYN flood mitigation mode tests are applied to all new connection requests to that particular destination. Traffic to other destinations is not subject to the tests.
NTP Response per Destination	This counter establishes a maximum packet rate threshold for NTP Responses to any single destination. A rate that exceeds the Threshold is treated as an NTP per Destination flood attack event. When the NTP per Destination limits are exceeded for a particular destination, NTP Response Packets are rate-limited to that destination without affecting NTP traffic to any other destination.

Continuous learning and adaptive thresholds

Most NBA systems use fixed value thresholds. Traffic, however, is never static. It shows trends and seasonality (a predictable or expected variation).

FortiDDoS uses adaptive thresholds. Adaptive thresholds take into account the traffic’s average, trend, and seasonality (expected or predictable variations).

Traffic prediction

Unlike other network behavior analysis (NBA) systems, FortiDDoS-F never stops learning. It continuously models inbound and outbound traffic patterns for key Layer 3, Layer 4, and Layer 7 parameters.

FortiDDoS-F uses the following information to model normal and abnormal traffic:

• The historical base, or weighted average, of recent traffic (more weight is given to recent traffic)
• The trend, or slope, of the traffic
• The seasonality of traffic over historical time periods

FortiDDoS-F uses these statistics to create a forecast for the next traffic period.

Traffic is non-deterministic; therefore, the forecast cannot be exact. The extent to which an observed traffic pattern is allowed to exceed its forecast is bounded by thresholds. Generally speaking, a threshold is a baseline rate that the system uses to compare observed traffic rates to determine whether a rate anomaly is occurring.

The FortiDDoS system maintains multiple thresholds for each key Layer 3, Layer 4, and Layer 7 parameter:

• Configured minimum threshold
• Estimated threshold
• Adaptive limit maximum threshold
• Adjustments for proxy IP addresses
• Packet count multipliers applied to traffic associated with an attack

The figure below illustrates how the system maintains multiple thresholds. The sections that follow explain the significance of each.

Adaptive, minimum and fixed

Configured minimum thresholds

The configured minimum threshold is a baseline of normal counts or rates. The baseline can be generated (based on statistics collected during the learning period) or stipulated (based on defaults or manually configured settings).

The configured minimum threshold is a factor in setting rate limits, but it is not itself the rate limit. Rate limits are set by the estimated threshold, a limit that is subject to heuristic adjustment based on average, trend, and seasonality.

Many of the graphs in the Monitor menu display the configured minimum threshold as a reference.

The table below summarizes the alternative methods for setting the configured minimum threshold.

Minimum threshold configuration settings

Settings	Usage
Thresholds	The thresholds configuration is open. You can set user-defined thresholds and fine-tune them. You can set reasonable values for port and protocol thresholds based on your knowledge of your network’s services and server capacity. It is advised that you have a thorough understanding of FortiDDoS before manually setting values for scalar thresholds.
System Recommendation	The recommended method for setting the configured minimum thresholds. The configured minimum thresholds are a product of the observed rates adjusted by a percentage that you specify.
Emergency Setup	Use if you do not have time to use Detection Mode to establish a baseline.
Factory Defaults	Use to quickly restore the system to high values. The factory defaults are high to avoid possible traffic disruption when you first put the system inline. In general, these settings are used together with Detection Mode when you are setting an initial baseline or a new baseline.
Percent Adjust	Use when you expect a spike in legitimate traffic due to an event that impacts business, like a news announcement or holiday shopping season.

Estimated thresholds

The estimated threshold is a calculated rate limit, based on heuristic adjustments.

The system models an adjusted normal baseline based on average, trend, and seasonality. It uses the heuristics to distinguish attack traffic from increases in traffic volume that is the result of legitimate users accessing protected resources.

The minimum value of an estimated threshold is the configured minimum threshold. In other words, if it is not predicting normal traffic becoming heavier than the baseline, it allows a rate at least as high as the configured minimum threshold.

The maximum value of an estimated threshold is the product of the configured minimum threshold and the adaptive limit. In other words, the system does enforce an absolute maximum rate limit.

Adaptive limit

The adaptive limit is a percentage of the configured minimum threshold.

An adaptive limit of 100% means no dynamic threshold estimation adjustment takes place once the configured minimum threshold is reached (that is, the threshold is a fixed value).

The product of the configured minimum threshold and adaptive limit is the absolute maximum rate limit. If the adaptive limit is 150% (the default), the system can increase the estimated threshold up to 150% of the value of the configured minimum threshold.

There are scenarios where FortiDDoS-F drops legitimate traffic because it cannot adapt quickly enough to a sudden change in traffic patterns. For example, when a news flash or other important announcement increases traffic to a company’s website. In these situations, you can use the Protection Profiles > Thresholds > Percent Adjust configuration page to increase all configured thresholds by a specific percentage.

Adjustments for proxy IP addresses

FortiDDoS can take account of the possibility that a source IP address might be a proxy IP address, and adjust the threshold triggers accordingly. If a source IP address is determined to be a proxy IP address, the system adjusts thresholds for a few key parameters by a factor you specify on the Global Settings > Proxy IP page.

Packet count multipliers applied to traffic associated with an attack

Packet count multipliers are adjustments to counters that are applied to traffic associated with an attack so that the thresholds that control drop and block responses are triggered sooner. You can configure multipliers for the following types of traffic:

• Source floods—Traffic from a source that the system has identified as the source of a flood.
• Layer 7 floods—Traffic for attacks detected based on a URL or Host, Referer, Cookie, or User-Agent header field.

You can use the Protection Profiles > Settings page to specify packet count multipliers.

When both Source flood and Layer 7 flood conditions are met, the packet count multipliers are compounded. For example, when there is a User Agent flood attack, a source is sending a User-Agent that is overloaded. If the Source multiplier is 4 and the Layer 7 multiplier is 64, the total multiplier that is applied to such traffic is 4 x 64 = 264. In effect, each time the source sends a Layer 7 packet with that particular User-Agent header, FortiDDoS considers each packet the equivalent of 256 packets.

Hierarchical nature of protocols and implication on thresholds

An HTTP packet has Layer 7, Layer 4 (TCP), and Layer 3 (IP) properties. A packet must be within the estimated thresholds of all these counters in order to pass through the FortiDDoS-F gateway. When it sets recommended thresholds, the system takes account of this complexity. If you set thresholds manually, you must also be sure that Layer 7 rates are consistent with Layer 4 and Layer 3 rates.

Protocol hierarchy for determining thresholds

The below illustrates system processing for an HTTP packet.

HTTP packet properties

The following IPv4/IPv6 packet properties are tracked:

• Protocol
• Fragment or not a fragment
• Source IP address (the system can monitor the packet rate from that specific source)

The following TCP packet properties are tracked:

• Destination port
• SYN or not a SYN packet
• TCP connection tuple (the system can monitor the packet rate within that connection)

An HTTP packet has the following properties that can be tracked:

• Method (GET, HEAD, POST, PUT, TRACE DELETE, OPTIONS, CONNECT)
• Method per Source
• URL
• Headers

The following IPv4/IPv6 packet properties are tracked:

• Protocol
• Fragment or not a fragment
• IP option values
• Source IP address, and hence packet rate from that specific source

The following UDP packet properties are tracked:

• Destination port 1-65, 535 (Port 0 is invalid)
• Source port 1-9999, plus 32 user-entered ports

A DNS message has the following additional properties that can be tracked in queries and responses:

• 24 DNS header, Query, Response and exploit anomalies
• Rates for Queries, MX, ALL, ZT, Qcount, Query/Source, Suspicious Sources, DNS Fragments
• DNS Response Codes 0-15

To summarize, because determining thresholds is a hierarchical process, avoid setting low thresholds on common conditions that can cause FortiDDoS-F to block legitimate traffic as well as attack traffic. The more specific you are about the type of traffic you want to allow as ‘normal’, the more effective the FortiDDoS-F is in blocking other traffic.