System Recommendations

System Recommendation thresholds are the heart of FortiDDoS operation. For long-term successful mitigation of all kinds of DDoS attacks, you must use the following process to create effective mitigation:

1. Create Service Protection Profile Rules (SPPs) and Configure Protection Subnets
2. Allow the system to learn traffic patterns for at least 1 week
3. Create Traffic Statistics reports for each SPP rule
4. Create System Recommendation Thresholds (below)

Failing to follow these steps may require a remediation process, which will take significantly longer than the original setup. If you have not done so, please complete steps 1-3 before attempting System Recommendation.

System Recommendation uses the Traffic Statistics previously generated to create about 264,000 thresholds per Service Protection Profile in each direction. Some key points:

• It is impossible to set 264,000 parameters manually. Hence the automation of System Recommendation thresholds. As a comparison, Emergency Setup sets only 3 thresholds that are relevant to inbound DDoS attacks.
• It would be extremely difficult to manage changes on 264,000 parameters after the thresholds are set. For this reason, FortiDDoS groups similar, contiguous parameters such as groups of TCP Ports, UDP Ports, Protocols and ICMP Types and Codes into ranges. For example, TCP Ports between 7000 and 8000 are not used often and may show no traffic during the Learning Period. If so, a single threshold value is created for the entire range of ports between 7000 and 8000. Each port has a threshold but they are all the same after System Recommendation is complete. Thresholds and ranges may be changed manually at any time but the System Recommended ranges and thresholds have been developed over many years of DDoS mitigation, usually making need for manual threshold changes unnecessary.
• FortiDDoS also associates traffic in both directions to the “Service Ports” under 10,000. Ephemeral Ports above 10,000 should show little traffic.
• If needed, FortiDDoS will create up to 512 ranges for each of the parameters above. These number of ranges are not often created, but if you see more than 40 to 50 ranges for a particular parameter, you may want to modify the default Low Traffic Thresholds described below. See below or contact FortiCare for support with these changes.

We strongly recommend you use the System Recommendation feature to set thresholds. The system recommendation procedure sets the configured minimum threshold to:

• A multiple (entered as a percentage and normally the default seen above) of the learned rates generated from the Traffic Statistics algorithm. OR
• A minimum Low Traffic threshold, whichever is higher

Note: If Traffic Statistics have not been completed for an SPP and a matching time-period, the System Recommendation operation is not allowed for the same. Be sure the period selector dropdown for System Recommendation matches the period you used to create Traffic Statistics.

You can use the Service Protection > Service Protection Policy > {SPP Rule} > Threshold Settings > System Recommendation tab to set change the multiplier and the minimum Low Traffic threshold for OSI Layers 3-7 and various parameter groups within Layer 4. These are typically not changed from the defaults, but if they are changed, Changes to these settings are persistent and saved to config. You can change these settings at any time and re-Save System Recommendations for the selected Traffic Statistics. Changed System Recommendations take effect immediately which might impact mitigation. Change the SPP operating Mode to Detection before changing System Recommendations. Then return the SPP to Prevention Mode if there are no unexpected drops in the logs.

If you have manually changed any Thresholds and then recreate System Recommendation Thresholds, the manual Thresholds will be overwritten.

Note: At the top of the System Recommendations page is an option (default enabled) to Set Outbound Thresholds To Max. Most users should leave this enabled since outbound attacks are rare and normally appear in "default" SPP rule. Managing outbound Thresholds and ranges is unnecessary. With this option enabled all outbound Thresholds for all parameters are set to system maximums. If a user wishes to set Outbound Thresholds based on Traffic Statistics reports, uncheck Set Outbound Thresholds To Max and Save to create Thresholds in both directions.

The resulting configured minimum thresholds are populated on the Service Protection > Service Protection Policy > {SPP Rule} > Thresholds tab. As you become a FortiDDoS expert, you can further tune the thresholds on that page.

Note: In the explanations below, System Recommendations sets some Thresholds to system maximums or does not set them at all, which has the same effect. These Thresholds are either rarely used for special circumstances or are covered by other mitigations. For example, you do not normally want to block Layer 3 Protocol 6 (TCP) or Protocol 17 (UDP) just because that protocol traffic is high, so it is not included in Thresholds. There are many other parameters above Layer 3 that will detect and mitigate attacks. However, if you are protecting web servers that never see UDP traffic you can manually set a very low UDP Protocol 17 threshold, which provides better protection from UDP-based attacks.

How the System Recommendation feature sets thresholds for different L3, L4 and L7 parameters

Thresholds are set to either the Traffic Statistics' maximum rate seen for that parameter over the period of the report, multiplied by the Layer 3, Layer 4 or Layer 7 Percentage shown on the page, or to the Low Traffic threshold, whichever is higher.

For example, if the Traffic Statistics for a TCP port (a Layer 4 parameter) are 100pps, the System Recommendations first multiplies that by the amount displayed in Layer 4: TCP > TCP Port Percentage (default 200% or 2x). 100 pps x 2 = 200 pps.

It then compares this 200 pps rate with the TCP Port Low Traffic rate, lower in the table (default 500 pps). Since the multiplied traffic statistics rate of 200 pps is lower than 500 pps, that port gets a System Recommended Threshold of 500 pps. If another TCP Port has a Traffic Statistics rate of 1000 pps, that is multiplied by 2, to 2000 pps which is higher than the Low Traffic threshold of 500 pps so that port threshold is set to 2000 pps.

When all port calculations are complete, the system then groups contiguous ports with similar packet rates into single ranges. For example, TCP ports between 6000 and 7000 are seldom used and may all have 500 pps thresholds. The system will create and display a single range for TCP Ports 6000-7000 with a threshold of 500 pps.

The system continues with Traffic Statistics multipliers or Low Traffic threshold and ranges through all parameters for the SPP. This process takes a few seconds.

Note, TCP and UDP Port calculations and ranges purposely omit some ports such as UDP 53 and TCP 80 and 443 (among others). FortiDDoS does not want to rate limit traffic to these ports solely on the port data rate when other parameters like SYNs, DNS Queries, SSL Renegotiation will provide more granular mitigation. You can manually add Thresholds for missing ports, but this is usually not required.

Once complete, Thresholds can be found on Service Protection > Service Protection Policy > {SPP Rule} > Thresholds with various sets of parameters shown on their own pages. Thresholds can be manually changed, added or deleted from those pages.

Note: DDoS attacks are not subtle and typically will be 100’s of thousands of pps. Setting a default low threshold to 1000 or even 10,000 will have little impact on mitigation. If in doubt contact Fortinet for advice.

Threshold Group	Notes
General	Depending on the Release, none, some, or ALL Outbound Thresholds may be set to system maximums (no Threshold-based mitigation even if the Outbound Direction is set to Prevention Mode). This can be changed on a per-SPP rule basis by disabling Service Protection Profiles > {SPP Rule} > Threshold Settings > System Recommendations: Set Outbound Thresholds To Max checkbox before creating System Recommendations. DDoS Attacks.
Scalars	Some Thresholds are not set by System Recommendation (equivalent to system maximums), because they are used in specific applications only. They may be set manually if needed. Layer 3/4 Scalar Thresholds: Most Active Destination, New Connections Layer 7 Scalar Thresholds: DNS Query per Source, DNS Packet Track per Source (Suspicious Sources on Monitor Graphs) Note: All Outbound thresholds are set to Max if Traffic Stats generated Empty data
DNS R-Code	Layer 7 Scalar DNS R-code Thresholds: These thresholds are manually created and intended for use only in systems where the DNS mitigation feature Match Response With Queries (DQRM) will not work: FortiDDoS sees only part of the traffic in an asymmetric network FortiDDos sees Encrypted DNS packets from Firewalls, Web Proxies, Email servers or other devices doing web/domain filtering via vendor services. DNS Response Code thresholds can be determined by viewing and recording the peak rate of RCode(0-15) over a period of time, usually 1 week or 1 month. (Traffic Monitor > Layer 3/4/7 > Layer 7 > DNS > DNS Response Code).
Fragment	Fragments from all protocols will be learned by Traffic Statistics as above. The learned traffic will be multiplied by the Layer 3 percentage in System Recommendations (our use the default low threshold) but the resulting Threshold will be applied to three different scalar parameters: Fragment –monitoring the rate of fragmented packets from Protocols other than TCP and UDP. TCP Fragment – monitoring the rate for TCP fragmented packets. UDP Fragment – monitoring the rate for UDP fragmented packets. Fragmented packets are a very common attack type but be careful with these Thresholds. Working from home has increased the level of Protocol 50 and UDP fragments because: Home routers that work with IPSEC (Protocol 50 / ESP) often do not support Path MTU and do not reduce the packet size before IPSEC headers are added, resulting in higher fragmented packet rates Home routers that don’t support IPSEC are forced to encapsulate the TCP or UDP packets in IPSEC and then encapsulate them again in UDP over Port 4500 (IPSEC NAT traversal), adding more bytes to the packet and again causing significant fragmentation.
Protocol	The system recommendation sets Thresholds and ranges for all 256 Protocols except for Protocol 6 (TCP) and Protocol 17 (UDP). Special case to consider: In the context of new Mirai floods which target all 65,535 UDP ports, it is useful to set a UDP Protocol "backup" threshold: Go to Traffic Monitor > Layer 3/4/7 > Layer 3 > Protocols and set the Protocol field to 17. Observe the peak rate over the last week or month. Multiply that rate by 5 and record it. if you see very large, infrequent spikes (50kpps or more), ignore these and look for the peak traffic without those spikes. Next, go to Service Protection Policy > {SPP Rule} > Thresholds > Protocols. Add a new range as "Proto_17" or similar. Set Protocol Start and End to ""17"". Set the Inbound Threshold to the number you recorded above. Leave the Outbound Threshold at default. Save the Range Policy. It will appear at the bottom of the list - list order is not important to the system.
ICMP Type/Code	There are 65,536 allowable ICMP Types and Codes. However, less than 150 are valid for IPv4 and IPv6 traffic. You may create ICMP Profile with ICMP Type Code Anomaly enabled and link to SPP rule to remove any unused ICMP Types and Codes as anomalies. Whether all 65k types/codes or the reduced valid set is used, FortiDDoS will set thresholds for all 65,536 Types and Codes. Contiguous ICMP type/codes that have the similar inbound and/or outbound traffic rates are grouped into ranges. A maximum of 512 ranges will be created to reduce management complexity. Normally only a few ranges are needed. Very few ICMP Types and Codes should show packet rates above default. If you see more than 4 or 5 ranges, contact Fortinet Support for advice. Please note that on all platforms, a single threshold entry is created for ICMP Type Code range 40:0-255:255 with threshold determined by recording the peak rate in that range.
TCP Port	Ports 0-10239 will be grouped into a series of ranges for ports with similar traffic. A single threshold is set for all ports above 10240. By default ALL Outbound TCP Port Thresholds will set to System Maximums (no outbound Threshold-based mitigation, even if Outbound direction is in Prevention Mode). If you need outbound mitigation, Disable Protection Profiles >{SPP Rule} > Threshold Settings > System Recommendations: Set Outbound Thresholds To Max checkbox before creating System Recommendations. The System Recommendation procedure does not set the threshold for widely used TCP service ports 20-23, 25, 80, 110, 143 and 443 because there are more granular TCP L4-L7 parameters to detect floods. The thresholds for these ports are not shown – you will see gaps in the Port Threshold ranges. These ports are internally set to high values. If these ports are not in use for a specific SPP, you can add thresholds or ACL the ports. You can identify if these ports are in use by examining the Traffic Statistics in the GUI. The system does not set the threshold for user-configured HTTP & SSL service ports (Service Protection > Service Protection Policy > {SPP Rule} > Service Port Settings). The thresholds for these are set to high values and will also be missing from the Port ranges as above. The system recommendation procedure does not set an Inbound or Outbound threshold for TCP port 53 because there are more granular DNS TCP parameters to detect floods. In reports and logs all traffic to or from 'service ports' 0-10239 or user-defined HTTP & SSL ports are associated with that port. Ephemeral high ports are not included in this traffic as they are generally irrelevant to protection of the service ports. If traffic is seen where both the Source Port and Destination Port are > 10240, the Destination Port will be shown on graphs and logs. A maximum of 512 different port ranges will be defined by the System Recommendation Algorithm. Notes: Because of the way the Algorithm operates, the Low Traffic Thresholds (default 500) may be changed slightly when System Recommendations are created. This is normal and does not affect mitigation. Managing 512 TCP Port ranges (52 pages on the GUI) can be difficult. Please see the instructions below to reduce Port ranges if needed.
UDP Port	The system recommendation procedure does not set an Inbound or Outbound threshold for UDP port 53 and UDP port 123 because there are more granular DNS UDP and NTP parameters to detect floods. The system sets system maximum Outbound Thresholds for UDP ports < 10240.A single Threshold is set for all ports >10239. In reports and logs, all traffic to or from UDP 'service ports' 0-10239. Ephemeral high ports are not included in this traffic as they are generally irrelevant to protection of the service ports. If traffic is seen where both the Source Port and Destination Port are >10239, the Destination Port will be shown on graphs and logs. Notes: UDP Port 53 is not assigned to a range. The expectation is that other parameters like Query thresholds will protect this port. You can manually create a threshold if you wish. A maximum of 512 different port ranges will be defined by the System Recommendation algorithm. Managing 512 UDP Port ranges (52 pages on the GUI) can be difficult. Please see the instruction below to reduce the number of ranges if needed.
HTTP Method per Source	The aggregate of all 8 Methods sent per Source is tracked for up to 4 million sources depending on model. FortiDDoS does not decrypt HTTPS so HTTP Methods per Source may not be seen.
HTTP Methods	Thresholds are set to either the observed maximum multiplied by the Layer 7 percentage, or to the Low Traffic threshold, whichever is higher, for all 8 HTTP Methods. FortiDDoS does not decrypt HTTPS so Methods may not be seen.
HTTP URL, Host, Cookie, Referer, User-Agent	The rate meters for URLs and HTTP headers are based on indexes. For each SPP in the HTTP Header, FortiDDoS indexes on the first 2047 characters of up to 64,000 URLs, 512 Hosts, 512 Cookies, 512 Referers and 512 User Agents. Packet rates vary across these indexes, SPPs, and traffic direction, depending on the time the baseline is taken. The “observed maximum” used by the system recommendation procedure is the peak packet rate for all indexes (excluding indexes with zero traffic). Thresholds are set to either the observed maximum multiplied by the Layer 7 percentage, or to the low traffic threshold, whichever is higher. FortiDDoS does not decrypt HTTPS so URL, Host, Cookie, Referer and User-Agent traffic may not be seen.

Before you begin:

• You must have generated traffic statistics for a learning period – Service Protection > Service Protection Policy > {SPP Rule} > Threshold Settings > System Recommendation > Generate Statistics. Ensure that the traffic statistics report that you generate for use with System Recommendation is for a period that is long enough to be a representative period of activity. A rule of thumb is 1 week for enterprise deployments and 1 month for ISP deployments – longer is always better. Shorter periods will usually require additional tuning. If necessary, reset statistics for the SPP before initiating the learning period.

• You must have Read-Write permission for Protection Profile settings.

To generate the system recommended thresholds:

1. Go to Service Protection > Service Protection Policy > {SPP Rule} > Threshold Settings > System Recommendation
2. Select the time period you wish to use for Traffic Statistics. If Apply to Thresholds button is disabled, there are no Traffic Statistics generated for the period you have selected. You will see system default System Recommendation settings for L3, L4 and L7 percentage multipliers used to increase the Thresholds above the accrual Traffic Statistics. You will also see default low traffic Thresholds for low traffic parameters. If you are an expert, you can adjust the System Recommendation settings as described in the table below. The adjusted settings will be persistent. They will not return to default after use.
3. Complete the configuration as described in the table.

   Settings	Guidelines
   Layer 3/4/7 Percentage	Multiply the generated Traffic Statistics by the specified percentage to compute the recommended thresholds. For example, if the value is 300%, the threshold is three times the Traffic Statistics learned rate. The default adjustment for the various layers is: Layer 3 = 300(%) Layer 4 = 200(%) Layer 7 = 200(%) Valid range is 100 (% - no change) to 500 (% - 5x the Traffic Statistics rate) Most users should not change these settings from default. Expert users may change these carefully.
   Layer 3/4/7 Low Traffic Threshold	Specify a minimum threshold to use instead of the recommended rate when the recommended rate is lower than this value. This setting is helpful when you think that the generated maximum rates are too low to be useful. The default is 500 with the following exceptions: • All Scalars and all UDP ports <10239 have their Outbound Thresholds automatically set to system maximum rates, to avoid outbound false-positive drops. These thresholds can be modified after creation if necessary by expert users. • Changing the low traffic threshold from default 500 is may be required for high traffic users with a lot of TCP and UDP Port traffic, in order to decrease the number of ranges. See the instructions below, if needed. • Changes to the low traffic threshold are persistent and will not revert to defaults after use. For example, if the generated maximum packet rate for inbound Layer 4 TCP packets is 2,000 and the outgoing rate is 3,000. The value of Layer 4 percentage is 300 (percent) and the value of Layer 4 low traffic threshold is 8,000. In this example: The recommended threshold for inbound packets is 6,000 (2,000 * 300% = 6,000). However, because 6,000 is less than the low traffic threshold of 8,000, the system sets the threshold to 8,000. The recommended threshold for outbound packets is 9,000 (3,000 * 300% = 9,000). Since 9,000 is greater than the low traffic threshold of 8,000, the system sets the threshold to 9,000.

4. Click Apply To Thresholds to generate the system recommended thresholds. Changes take effect immediately.
5. Go to Service Protection > Service Protection Policy > {SPP Rule} > Thresholds and review the thresholds.

Adjusting the system recommended thresholds

From step 5 above, Look specifically at TCP and UDP Port thresholds on Service Protection > Service Protection Policy > {SPP Rule} > Thresholds. If number of Port Threshold Ranges is more than 40, it is advisable to change the Low Traffic Threshold for this parameter to reduce the number of ranges.

You will notice in Service Protection > Service Protection Policy > {SPP Rule} > Threshold Settings > System Recommendation, there are separate Low Traffic Thresholds for:

• Layer 4 Scalars and ICMP
• TCP Ports
• UDP Ports

If, for example, the UDP Port page shows 100 ranges, then return to System Recommendations and double the UDP Ports Low Traffic Threshold without changing any other settings. Save this and check the UDP Port Thresholds page again to see if ranges are now under 40. If not, double the UDP Ports Low Traffic Threshold again and check again until ranges are under 40. Do the same with TCP ports. Other parameters should not create large numbers of ranges.

To configure using the CLI: config ddos spp rule edit <spp_name> set system-recommendation enable set threshold-system-recommended-report-period {1-hour | 8-hours | 1-day | 1-week | 1-month | 1-year | 10-min} set threshold-system-recommended-layer-3-low-traffic <integer> set threshold-system-recommended-layer-3-percentage <integer> set threshold-system-recommended-layer-4-low-traffic <integer> set threshold-system-recommended-layer-4-percentage <integer> set threshold-system-recommended-layer-4-tcp-port-low-traffic <integer> set threshold-system-recommended-layer-4-tcp-port-percentage <integer> set threshold-system-recommended-layer-4-udp-port-low-traffic <integer> set threshold-system-recommended-layer-4-udp-port-percentage <integer> set threshold-system-recommended-layer-7-low-traffic <integer> set threshold-system-recommended-layer-7-percentage <integer> next end