Custom Policy

What if you want to allow a web crawler, but only if it is not too demanding, and comes from a source IP that is known to be legitimate for that crawler? What if you want to allow only a client that is a senior manager’s IP, and only if it hasn’t been infected by malware whose access rate is contributing to a DoS?

custom rules provide a degree of flexibility for these types of complex conditions. You can combine any or all of these criteria:

• Source IP
• User
• Rate limit (including rate limiting for specific types of content)
• HTTP header or response code
• URL
• Transaction or packet interval timeout
• Geo IP
• Parameter
• Time period

You use the rule's filters to specify all criteria that you require allowed traffic to match.

The filters apply to request traffic only, with the following exceptions:

• HTTP Response Code and Content Type apply to responses.
• Signature Violation applies to either requests or responses, depending on which signatures you enable.
• Occurrence applies to either requests or responses.

FortiWeb includes predefined rules that defend against some popular attacks. You cannot edit these predefined rules, but you can view their settings or create duplicates of them that you can edit (that is, by cloning).

Advanced access control is available even if FortiWeb derives client source IP addresses from the X-header field. For details, see Defining your proxies, clients, & X-headers.

To configure an advanced access control rule

1. Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Rule tab.
   To access this part of the web UI, your administrator’s account access profile must have Read and Write permission to items in the Web Protection Configuration category. For details, see Permissions.
2. Do one of the following:
   

• To create a new rule, click Create New.
• To create a new rule based on a predefined rule, select the predefined rule to use, and then click Clone.

If you are cloning a predefined rule, enter a name for your new rule, and then click OK. To edit or review the rule settings, select the rule, and then click Edit.

Configure these settings:

Name	Type a unique name that can be referenced in other parts of the configuration. The maximum length is 63 characters.
Action	Select which action the FortiWeb appliance will take when it detects a violation of the rule: Alert—Accept the request and generate an alert email and/or log message. Alert & Deny—Block the request (or reset the connection) and generate an alert email and/or log message. You can customize the web page that FortiWeb returns to the client with the HTTP status code. For details, see Customizing error and authentication pages (replacement messages). Deny (no log)—Block the request (or reset the connection). Period Block—Block subsequent requests from the client for a number of seconds. Also configure Block Period. You can customize the web page that FortiWeb returns to the client with the HTTP status code. For details, see Customizing error and authentication pages (replacement messages). The default value is Alert. Caution: This setting is ignored when Monitor Mode is enabled. Note: Logging and/or alert email will occur only if enabled and configured. For details, see Logging and Alert email.
Block Period	Type the number of seconds that you want to block subsequent requests from the client after the FortiWeb appliance detects that the client has violated the rule. This setting is available only if Action is set to Period Block. The valid range is from 1 to 3,600 seconds (1 hour). For details, see Monitoring currently blocked IPs.
Severity	When rule violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level the FortiWeb appliance will use when it logs a violation of the rule: Informative Low Medium High The default value is Medium.
Trigger Action	Select which trigger, if any, that the FortiWeb appliance will use when it logs and/or sends an alert email about a violation of the rule. For details, see Viewing log messages.
Bot Confirmation	Enable to confirm if the client is indeed a bot.
For Browser
Verification Method	Disabled: Not to carry out the real browser, CAPTCHA, and reCAPTCHA verification. Real Browser Enforcement—Specifies whether FortiWeb returns a JavaScript to the client to test whether it is a web browser or automated tool when it meets any of the specified conditions. If the client fails the test or does not return results before the Validation Timeout expires, FortiWeb applies the Action. If the client appears to be a web browser, FortiWeb allows the client to exceed the action. CAPTCHA Enforcement—Requires the client to successfully fulfill a CAPTCHA request. If the client cannot successfully fulfill the request within the Max Attempt Times or doesn't fulfill the request within the Validation Timeout, FortiWeb applies the Action and sends the CAPTCHA block page. For details, see Customizing error and authentication pages (replacement messages). CAPTCHA verification will not pop out for the bot confirmation again for the same user within 10 mins timeout. reCAPTCHA Enforcement—Requires the client to successfully fulfill a reCAPTCHA request. If the client cannot successfully fulfill the request within the Validation Timeout, FortiWeb applies the Action and sends the CAPTCHA block page. For details, see Customizing error and authentication pages (replacement messages). Please note that the bot confirmation methods don't work with the filters for the response packets. For example, the system won't carry out CAPTCHA Enforcement even if a request triggers an HTTP response that matches the HTTP Response Code filter, and it also won't take any action on this packet. Therefore, it's strongly recommended not to enable Bot Confirmation for the response packet filters.
reCAPTCHA	Select the reCAPTCHA server you have created in the reCAPTCHA Server tab in User > Remote Server. See Creating reCAPTCHA servers
Validation Timeout	Enter the maximum amount of time (in seconds) that FortiWeb waits for results from the client. Available only when the Verification Method is Real Browser Enforcement, CAPTCHA Enforcement, or reCAPTCHA Enforcement.
Max Attempt Times	If CAPTCHA Enforcement is selected for Verification Method, enter the maximum number of attempts that a client may attempt to fulfill a CAPTCHA request.
For Mobile Client App	Available only when Mobile Application Identification is enabled in System > Config > Feature Visibility.
Verification Method	Disabled: Not to carry out the mobile token verification. Mobile Token Validation: Requires the client to use mobile token to verify whether the traffic is from mobile devices. To apply mobile token validation, you must enable Mobile App Identification in Web Protection Profile.

Click OK.

Click Create New to add an entry to the set.

From Filter Type, select one of the following conditions that a request must match in order to be allowed, then click OK.

Filters of the different types are in "AND" relationship. However, filters of the same type are in "OR" relationship, which means any packet that hits either one of them will be considered as a match. This is tricky when the filters are set with "not match" conditions. For example, in order to block the source IPs that are not in a certain IP ranges, you set the following two Source IP filters: Source IP Filter A: Source IP does not match 10.254.226.0 -10.254.227.254 Source IP Filter B: Source IP does not match 10.254.228.0 -10.254.229.254 But in fact these two filters together make the source IP check invalid, because IPs in range A meet the condition in Filter B, and likewise for IPs in range B. As a result, IP addresses in range A or B will all be considered as a match, which is contradictory to the original purpose of letting these packets go. This is a logic loophole. In later release we will support adding multiple IP ranges in a single filter so that such purpose can be fulfilled.

• Source IPv4/IPv6/IP Range—Type the IP address of a client that is allowed. Depending on your configuration of how FortiWeb derives the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

  To enter an address range, enter the first and last address in the range separated by a hyphen. For example, for an IPv4 address, enter 192.0.2.1 – 192.0.2.155. For an IPv6 address, enter 2001::1-2001::100.

  For Meet this condition if, select one of the following:

  • Source IP matches—The request will match the condition if it contains the Source IPv4/IPv6/IP Range value.
  • Source IP does not match—The request will match the condition if it doesn't contain the Source IPv4/IPv6/IP Range value.

• User—Enter a user name to match, and then specify whether the condition matches if the request contains the specified user name or matches only for user names other than the specified one.

  Note: This type of filter requires you to select a user tracking policy in any protection profile that uses this advanced access policy. For details, see Tracking.

• URL—Enter a literal URL, such as /folder1/index.htm that the HTTP request must contain in order to match the rule, or use wildcards to match multiple URLs, such as /folder1/* or /folder1/*/index.htm. Or type a regular expression that matches one or more URLs, such as /index\.jsp. Do not include the host name.

To accept requests that do not match the URL, do not precede the URL with an exclamation mark (!). Use the CLI to configure the reverse-match {no | yes} setting for this filter. For details, see the FortiWeb CLI Reference: HTTPs://docs.fortinet.com/product/fortiweb/

• HTTP Header—Indicate a single HTTP Header Name such as Host:, and all or part of its value in Header Value. The request matches the condition if that header matches the exact name or value, or matches your regular expression (depending on whether you have selected Simple String or Regular Expression). Value matching is case sensitive and supports null value match.

  • If you enable Missing Header Name, the request matches the condition if it does not contain the specified header. Please note that this setting does not take effect for HTTP2 packets without the following headers:
    • :method
    • :scheme
    • :path
    • :authority
    • :status

    HTTP2 packets without the above headers will not go far to be scanned against the custom rule settings. It will be considered as illegitimate and be abandoned directly when it arrives at FortiWeb at the first place.

  • If you enable Header Empty Value Check, the request matches the condition if it contains the specified header but the value of the matched header is empty.
    Missing Header Name and Header Empty Value Check can't be enabled at the same time.
  • If you enable Header Value Reverse Match, the request matches the condition if the header does not contain the exact value or regular expression.
  • Optionally, enable HTTP Method Check and configure a simple string or regular expression for the HTTP method that FortiWeb will search for in the header field. When you enable HTTP Method Check, you can also enable HTTP Method Reverse Match so that the request matches the condition if the header does not contain the HTTP method's exact value or regular expression.

  • FortiWeb supports Misformatted Basic Scheme Check. It displays only when Predefined Header name and Authorization are selected, and Missing Header Name and Empty Header Value Check are disabled.

To prevent accidental matches, specify as much of the header’s value as possible. Do not use an ambiguous substring. For example, entering the value 192.0.2.1 would also match the IPs 192.0.2.10-19 and 192.0.2.100-199. This result is probably unintended. The better solution would be to configure either: a regular expression such as ^192.0.2.1$ or a source IP condition instead of an HTTP header condition

• Access Rate Limit—This is the number of requests per second per client IP. Depending on your configuration of how FortiWeb will derive the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

  You can add only one Access Rate Limit filter to each rule.

• Signature Violation—Matches if FortiWeb detects a selected category or list of attack signatures in the request or response. The following categories are available:
  
  • Cross Site Scripting
  • Cross Site Scripting (Extended)
  • SQL Injection
  • SQL Injection (Extended)
  • Generic Attacks
  • Generic Attacks (Extended)
  • Known Exploits
  • Trojans
  • Information Disclosure

  • Personally Identifiable Information

  • Bad Robot
  • Custom Signature (group or individual rule)
  A custom rule Vulnerability-Scanning is predefined, with some signature categories and lists customized.
  To use one of these categories in an advanced access control rule, enable the corresponding item in your signatures configuration. For details, see Blocking known attacks .

• Geo IP—Choose the countries to match. If you select Yes, FortiWeb matches the traffic from all countries except the ones you select. If you select No, FortiWeb matches the traffic from the countries you select.

• Transaction Timeout—Matches if the lifetime of a HTTP transaction exceeds the transaction timeout you specify. Specify a timeout value of 1 to 3600 seconds.
• HTTP Response Code—Matches if a HTTP response code matches a code or range of codes that you specify. For example, 404 or 500-503. To specify more than one response code or range, create additional HTTP Response Code filters.
  If Real Browser Enforcement is enabled in Verification Method, the HTTP Response Code filter can only work with code 200.
• Content Type—Matches an HTTP response for a file that matches one of the specified types. Use with Occurrence to detect and control web scraping (content scraping) activity.
• Packet Interval Timeout—Matches if the time period between packets arriving from either the client or server (request or response packets) exceeds the value in seconds you specify for Packet Timeout Interval. Enter a value from 1 to 60.
• Time Period—Matches if the time period of a request matches that you specify. You can set a daily period or fixed period.
• Occurrence—Matches if a transaction matches other filter types in the current rule at a rate that exceeds a threshold you specify.
  • To measure the rate by counting source client IP address, for Traced By, select Source IP.
  • To measure by HTTP session, select HTTP Session.

    Note: The HTTP Session option requires that you enable the Configuring a protection profile for inline topologies option in your protection profile. For details, see Configuring a protection profile for inline topologies.

  • To measure by client, select User.

    Note: The User option requires that you enable User Tracking in your protection profile. For details, see Configuring a protection profile for inline topologies.

  • To count the occurrence both by the hit times and the percentage, switch on Enable Percentage Matching, then enter the percentage. For example, if the occurrence is 5, and the percentage is 10%, then 5 or more hits out of 50 requests will be considered a match.

Click OK to exit the sub-dialog and return to the rule configuration.

Repeat the previous steps for each individual criteria that you want to add to the access rule.
For example, you can require both a matching request URL, HTTP header, and client source IP in order to allow a request.
You can add only one Access Rate Limit filter to each rule.

Click OK to save the rule.

Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Policy tab.

Click Create New. Group the advanced access rules into a policy.
For example, to create a policy that allows rate-limited access by 3 client IPs, you would group the corresponding 3 advanced access rules for each of those IPs into the policy.

Type a name for the custom policy which can be referenced in other parts of the configuration.

For Threat Weight, drag the bar to set the threat weight for each custom policy.

To apply the advanced access policy, select it as the Custom Policy in a protection profile. For details, see Configuring a protection profile for inline topologies or Configuring a protection profile for an out-of-band topology or asynchronous mode of operation.

Attack log messages contain Custom Access Violation when this feature detects an unauthorized access attempt.

See also

• IPv6 support