Combination access control & rate limiting

What if you want to allow a web crawler, but only if it is not too demanding, and comes from a source IP that is known to be legitimate for that crawler? What if you want to allow only a client that is a senior manager’s IP, and only if it hasn’t been infected by malware whose access rate is contributing to a DoS?

Advanced access control rules provide a degree of flexibility for these types of complex conditions. You can combine any or all of these criteria:

• Source IP
• User
• Rate limit (including rate limiting for specific types of content)
• HTTP header or response code
• URL
• Predefined or custom attack or data leak signature violation
• Transaction or packet interval timeout
• Real browser enforcement
• CAPTCHA enforcement

You use the rule's filters to specify all criteria that you require allowed traffic to match.

The filters apply to request traffic only, with the following exceptions:

• HTTP Response Code and Content Type apply to responses.
• Signature Violation applies to either requests or responses, depending on which signatures you enable.

FortiWeb includes predefined rules that defend against some popular attacks. You cannot edit these predefined rules, but you can view their settings or create duplicates of them that you can edit (that is, by cloning).

Advanced access control is available even if FortiWeb derives client source IP addresses from the X-header field. For details, see Defining your proxies, clients, & X-headers.

To configure an advanced access control rule

1. Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Rule tab.
   To access this part of the web UI, your administrator’s account access profile must have Read and Write permission to items in the Web Protection Configuration category. For details, see Permissions.
2. Do one of the following:
   

• To create a new rule, click Create New.
• To create a new rule based on a predefined rule, select the predefined rule to use, and then click Clone.

If you are cloning a predefined rule, enter a name for your new rule, and then click OK. To edit or review the rule settings, select the rule, and then click Edit.

Configure these settings:

Name	Type a unique name that can be referenced in other parts of the configuration. The maximum length is 63 characters.
Action	Select which action the FortiWeb appliance will take when it detects a violation of the rule: Alert—Accept the request and generate an alert email and/or log message. Alert & Deny—Block the request (or reset the connection) and generate an alert email and/or log message. You can customize the web page that FortiWeb returns to the client with the HTTP status code. For details, see Customizing error and authentication pages (replacement messages). Deny (no log)—Block the request (or reset the connection). Period Block—Block subsequent requests from the client for a number of seconds. Also configure Block Period. You can customize the web page that FortiWeb returns to the client with the HTTP status code. For details, see Customizing error and authentication pages (replacement messages). The default value is Alert. Caution: This setting is ignored when Monitor Mode is enabled. Note: Logging and/or alert email will occur only if enabled and configured. For details, see Logging and Alert email.
Block Period	Type the number of seconds that you want to block subsequent requests from the client after the FortiWeb appliance detects that the client has violated the rule. This setting is available only if Action is set to Period Block. The valid range is from 1 to 3,600 (1 hour). The default value is 60. For details, see Monitoring currently blocked IPs.
Severity	When rule violations are recorded in the attack log, each log message contains a Severity Level (severity_level) field. Select which severity level the FortiWeb appliance will use when it logs a violation of the rule: Informative Low Medium High The default value is Medium.
Trigger Action	Select which trigger, if any, that the FortiWeb appliance will use when it logs and/or sends an alert email about a violation of the rule. For details, see Viewing log messages.
Bot Recognition	You can select between: Real Browser Enforcement—Specifies whether FortiWeb returns a JavaScript to the client to test whether it is a web browser or automated tool when it meets any of the specified conditions. If the client fails the test or does not return results before the Validation Timeout expires, FortiWeb applies the Action. If the client appears to be a web browser, FortiWeb allows the client to exceed the action. For details, see Bot analysis. CAPTCHA Enforcement—Requires the client to successfully fulfill a CAPTCHA request. If the client cannot successfully fulfill the request within the Max Attempt Times or doesn't fulfill the request within the Validation Timeout, FortiWeb applies the Action and sends the CAPTCHA block page. For details, see Customizing error and authentication pages (replacement messages).
Max Attempt Times	If CAPTCHA Enforcement is selected for Bot Recognition, enter the maximum number of attempts that a client may attempt to fulfill a CAPTCHA request. The default value is 3. The valid range is 1–5.
Validation Timeout	Enter the maximum amount of time (in seconds) that FortiWeb waits for results from the client for Bot Recognition. The default value is 20. The valid range is 5–30.

Click OK.

Click Create New to add an entry to the set.

From Filter Type, select one of the following conditions that a request must match in order to be allowed, then click OK.

The Filter Type value determines which settings are displayed in the next dialog box.

• Source IPv4/IPv6/IP Range—Type the IP address of a client that is allowed. Depending on your configuration of how FortiWeb derives the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

  To enter an address range, enter the first and last address in the range separated by a hyphen. For example, for an IPv4 address, enter 192.0.2.1 – 192.0.2.155. For an IPv6 address, enter 2001::1-2001::100.

  For Meet this condition if, select one of the following:

  • Source IP matches—The request will match the condition if it contains the Source IPv4/IPv6/IP Range value.
  • Source IP does not match—The request will match the condition if it doesn't contain the Source IPv4/IPv6/IP Range value.

• User—Enter a user name to match, and then specify whether the condition matches if the request contains the specified user name or matches only for user names other than the specified one.

  Note: This type of filter requires you to select a user tracking policy in any protection profile that uses this advanced access policy. For details, see Tracking users.

• URL—Type a regular expression that matches one or more URLs, such as /index\.jsp. Do not include the host name.

To accept requests that do not match the URL, do not precede the URL with an exclamation mark (!). Use the CLI to configure the reverse-match {no | yes} setting for this filter. For details, see the FortiWeb CLI Reference: http://docs.fortinet.com/fortiweb/reference

• HTTP Header—Indicate a single HTTP Header Name such as Host:, and all or part of its value in Header Value. The request matches the condition if that header matches the exact name or value, or matches your regular expression (depending on whether you have selected Simple String or Regular Expression). Value matching is case sensitive and supports null value match.

  If you select Header Value Reverse Match, the request matches the condition if the header does not contain the exact value or regular expression.

  Optionally, enable HTTP Method Check and configure a simple string or regular expression for the HTTP method that FortiWeb will search for in the header field. When you enable HTTP Method Check, you can also enable HTTP Method Reverse Match so that the request matches the condition if the header does not contain the HTTP method's exact value or regular expression.

To prevent accidental matches, specify as much of the header’s value as possible. Do not use an ambiguous substring. For example, entering the value 192.0.2.1 would also match the IPs 192.0.2.10-19 and 192.0.2.100-199. This result is probably unintended. The better solution would be to configure either: a regular expression such as ^192.0.2.1$ or a source IP condition instead of an HTTP header condition

• Access Rate Limit—This is the number of requests per second per client IP. Depending on your configuration of how FortiWeb will derive the client’s IP, this may be the IP address that is indicated in an HTTP header rather than the IP header. For details, see Defining your proxies, clients, & X-headers.

  You can add only one Access Rate Limit filter to each rule.

• Signature Violation—Matches if FortiWeb detects a selected category of attack signature in the request or response. The following categories are available:
  
  
  • Cross Site Scripting
  • Cross Site Scripting (Extended)
  • SQL Injection
  • SQL Injection (Extended)
  • SQL Injection (Syntax Based Detection)
  • Generic Attacks
  • Generic Attacks (Extended)
  • Known Exploits
  • Custom Signature (group or individual rule)

• Geo IP—Choose the countries to match. If you select Yes, FortiWeb matches the traffic from all countries except the ones you select. If you select No, FortiWeb matches the traffic from the countries you select.

To use one of these categories in an advanced access control rule, enable the corresponding item in your signatures configuration. For details, see Blocking known attacks & data leaks.

• Transaction Timeout—Matches if the lifetime of a HTTP transaction exceeds the transaction timeout you specify. Specify a timeout value of 1 to 3600 seconds.
• HTTP Response Code—Matches if a HTTP response code matches a code or range of codes that you specify. For example, 404 or 500-503. To specify more than one response code or range, create additional HTTP Response Code filters.
• Content Type—Matches an HTTP response for a file that matches one of the specified types. Use with Occurrence to detect and control web scraping (content scraping) activity.
• Packet Interval Timeout—Matches if the time period between packets arriving from either the client or server (request or response packets) exceeds the value in seconds you specify for Packet Timeout Interval. Enter a value from 1 to 60.
• Occurrence—Matches if a transaction matches other filter types in the current rule at a rate that exceeds a threshold you specify.
  • To measure the rate by counting source client IP address, for Traced By, select Source IP.
  • To measure by HTTP session, select HTTP Session.

    Note: The HTTP Session option requires that you enable the Session Management option in your protection profile. For details, see Configuring a protection profile for inline topologies.

  • To measure by client, select User.

    Note: The User option requires that you enable User Tracking in your protection profile. For details, see Configuring a protection profile for inline topologies.

Click OK to exit the sub-dialog and return to the rule configuration.

Repeat the previous steps for each individual criteria that you want to add to the access rule.
For example, you can require both a matching request URL, HTTP header, and client source IP in order to allow a request.
You can add only one Access Rate Limit filter to each rule.

Click OK to save the rule.

Go to Web Protection > Advanced Protection > Custom Policy, and select the Custom Policy tab.

Click Create New. Group the advanced access rules into a policy.
For example, to create a policy that allows rate-limited access by 3 client IPs, you would group the corresponding 3 advanced access rules for each of those IPs into the policy. In Priority, enter the priority for each rule in relation to other defined rules. Rules with lower numbers (higher priority) are applied first.

To apply the advanced access policy, select it as the Custom Policy in a protection profile. For details, see Configuring a protection profile for inline topologies or Configuring a protection profile for an out-of-band topology or asynchronous mode of operation.

Attack log messages contain Custom Access Violation when this feature detects an unauthorized access attempt.

See also

• IPv6 support