Fortinet black logo

Handbook

Configuring an Advanced Bot Protection policy

Configuring an Advanced Bot Protection policy

Once your FortiADC is successfully connected to the FortiGuard Advanced Bot Protection server via the Advanced Bot Protection Fabric Connector, you can configure an Advanced Bot Protection policy for your virtual server to protect your online applications from malicious bots and automated attacks. By incorporating FortiGuard ABP into FortiADC's server policy, client traffic will be directed to the FortiGuard ABP service deployed on Google Cloud where it will be analyzed to identify any malicious bot behavior and initiate appropriate actions in response.

FortiGuard ABP features a multi-dimensional deep learning engine that learns and tracks bot attacks over time, delivering the highest possible accuracy of classification between humans, and good and bad bots.

FortiGuard ABP protects against a wide range of threats, including the following:

  • Data harvesting
  • Credential stuffing attacks
  • Account takeover attempts
  • DDoS attacks

No.

Description

1 User request reaches FortiADC (as Reverse Proxy).
2 FortiADC inserts a JavaScript to the HTTP/S response for telemetric information.
3 The client and FortiADC (via Fabric connector) share telemetry data (such as IP, headers, and device fingerprinting) with the Advanced Bot Protection engine.
4 Using Deep Learning, FortiGuard ABP determines if the client is a human or a bot.
5 FortiGuard ABP sends instructions back to FortiADC to initiate an action against the request (such as block, CAPTCHA, or allow).

After you have configured the Advanced Bot Protection policy, you can reference it in a WAF Profile and apply it in a virtual server policy. However, the Advanced Bot Protection policy does not activate until the FortiGuard ABP Application is fully analyzed and Pre-Provisioned to protect the Application.

Pre-Provisioning is required to identify all URLs that should be protected in your Application domain (such as login URLs), and the locations to which JavaScript need to be inserted to collect client information. Without these resources, FortiADC will not be able to insert the necessary JavaScript for bot detection.

Pre-Provisioning is triggered upon creating the Application, and requires 2 to 3 days to complete. During this process, your FortiGuard ABP Application will be in pending status until Pre-Provisioning is complete. When the Application status is ready, Advanced Bot Protection can be activated in your FortiADC.

When Advanced Bot Protection is initially activated, it is recommended to set the WAF action that would allow you to observe and log any events detected by the FortiGuard Advanced Bot Protection, instead of immediately setting to block.

FortiGuard Advanced Bot Protection uses a multidimensional deep learning engine to learn and track bot attacks over time by using sophisticated AI model training. As FortiGuard ABP builds its training model, it will continue to improve and refine its bot detection capabilities. However, this may mean triggering false positives in the initial stages of the AI model training.

Before you begin:
  • You must have enabled and successfully connected the Advanced Bot Protection connector on Security Fabric > Fabric Connectors.
    The Advanced Bot Protection module under Web Application Firewall is available only after the Advanced Bot Protection connector is enabled.
  • You must have Read-Write permission for Security settings.
  • You must have access to the FortiGuard Advanced Bot Protection User Portal to obtain the Application ID from an existing Application or create a new configuration. For more information, see Obtaining the Application ID from the FortiGuard ABP User Portal.

Decompression must be enabled to support JavaScript insertion, which is critical to FortiGuard Advanced Bot Protection functionality.

Ensure a Decompression policy is enabled in your HTTP/HTTPS Application Profile to allow JavaScript to be inserted into compressed HTTP/HTTPS web content. To increase performance, most websites utilize HTTP compression to reduce the size of transmitted data. This compressed HTTP/HTTPS content must be decompressed to allow FortiADC to insert the required JavaScript tag to the HTML. If the real server response is not compressed, then decompression is unnecessary. However, if the real server response is compressed then decompression must be enabled, otherwise the JavaScript will fail to insert.

To configure an Advanced Bot Protection policy:
  1. Go to Web Application Firewall > Advanced Bot Protection.
  2. In the Advanced Bot Protection tab, click Create New to display the configuration editor.
  3. Configure the following Advanced Bot Protection settings:

    Setting

    Description

    Name

    Specify a name for the Advanced Bot Protection policy.
    Valid characters are A-Z, a-z, 0-9, _, and -. No spaces. The configuration name cannot be edited once it has been saved.

    Status

    Enable/disable the status of this Advanced Bot Protection policy.

    Status must be enabled to display Advanced Bot Protection configuration options.

    Application ID

    Specify the Application ID assigned to your FortiGuard ABP Application.

    The Application ID is used to bind this Advanced Bot Protection policy to the FortiGuard ABP Application.

    For steps on how to obtain the Application ID from the FortiGuard ABP User Portal, see Obtaining the Application ID from the FortiGuard ABP User Portal.

    Action

    Specify a WAF action object to apply when a bot is detected. You can specify a predefined or user-defined WAF action profile. (See Configuring WAF Action objects.)

    Predefined WAF actions:

    • alert — WAF policies will allow the traffic to pass and log the event.
    • block — WAF policies will drop the current attack session by HTTP 403 message and block the attacker (according the attacker’s IP address) for 1 hour, and log the event.
    • captcha — WAF policies will allow the traffic to pass if the client successfully fulfills the CAPTCHA request, and log the event.
    • deny — WAF policies will the drop current attack session by HTTP 403 message, and log the event.
    • silent-deny — WAF policies will drop the current attack session by HTTP 403 message, without logging the event.

    The default action is alert.

    Severity

    Select the event severity to log when a bot is detected:

    • High — Log as high severity events.
    • Medium — Log as a medium severity events.
    • Low — Log as low severity events.

    The default is Low.

    Exception Name

    Select an exception configuration object. Exceptions identify specific hosts or URL patterns that are not subject to processing by this rule.

  4. Optionally, click Verify to validate the inputted Application ID against multiple parameters to ensure the connection between FortiADC and the FortiGuard ABP Application is successfully established.

    For more information, see Verifying the Application ID and connection status to FortiGuard ABP.
  5. Click Save.
    Once the Advanced Bot Protection policy is saved, you can reference it in a WAF Profile configuration.

It is strongly recommended to verify the Application ID and FortiGuard ABP server connection prior to completing the Advanced Bot Protection policy configuration. Even though this is an optional step, it is helpful to diagnose any potential issues and apply fixes early.

When the Advanced Bot Protection policy is created, an internal verification is automatically conducted to verify the status of the Application ID and FortiGuard ABP server connection. If the Application ID is not valid, or any other validation parameters has failed, the Advanced Bot Protection policy will fail to function and the system will log the failure to send the ABP policy.

Verifying the Application ID and connection status to FortiGuard ABP

The Advanced Bot Protection policy can only function if the Application ID is valid and the connection to the FortiGuard ABP server is successfully established. FortiADC validates multiple parameters, including if the Application ID is available, the FortiGuard ABP server connectivity, if the FortiGuard ABP license is valid. There are two types of messages as differentiated by text color: green text indicates a positive status where all required parameters are validated successfully; and red text that indicate one or more parameters did not pass validation.

The following table describes some common verification results.

Verification status message

Guidelines

Success (green)

All required parameters pass validation; application ID is available, FortiGuard ABP server certificate is valid, network connectivity is good, etc.

Application not found (red)

The Application ID does not exist. This could be an input error.

Account license invalid (red)

The FortiGuard ABP license is not valid. Please verify your license details or contact Fortinet Support.

Couldn't connect to server (red)

Unable to connect to the FortiGuard ABP server. Please check your network settings.

Couldn't resolve hostname (red)

Unable to resolve the hostname of the FortiGuard ABP server. Please check your network settings.

No available SN cert (red)

The device does not have an available SN certificate. Please check your local certificate.

No available CA cert (red)

The device does not have an available CA certificate. Please check your CA certificate.

Problem with the local certificate

An error occurred with the remote server certificate. Please check your local certificate.

SSL peer certificate or SSH remote key was not OK

An error occurred with the remote server certificate involving the SSL peer certificate or SSH remote key. Please check your local certificate.

Configuring an Advanced Bot Protection policy

Once your FortiADC is successfully connected to the FortiGuard Advanced Bot Protection server via the Advanced Bot Protection Fabric Connector, you can configure an Advanced Bot Protection policy for your virtual server to protect your online applications from malicious bots and automated attacks. By incorporating FortiGuard ABP into FortiADC's server policy, client traffic will be directed to the FortiGuard ABP service deployed on Google Cloud where it will be analyzed to identify any malicious bot behavior and initiate appropriate actions in response.

FortiGuard ABP features a multi-dimensional deep learning engine that learns and tracks bot attacks over time, delivering the highest possible accuracy of classification between humans, and good and bad bots.

FortiGuard ABP protects against a wide range of threats, including the following:

  • Data harvesting
  • Credential stuffing attacks
  • Account takeover attempts
  • DDoS attacks

No.

Description

1 User request reaches FortiADC (as Reverse Proxy).
2 FortiADC inserts a JavaScript to the HTTP/S response for telemetric information.
3 The client and FortiADC (via Fabric connector) share telemetry data (such as IP, headers, and device fingerprinting) with the Advanced Bot Protection engine.
4 Using Deep Learning, FortiGuard ABP determines if the client is a human or a bot.
5 FortiGuard ABP sends instructions back to FortiADC to initiate an action against the request (such as block, CAPTCHA, or allow).

After you have configured the Advanced Bot Protection policy, you can reference it in a WAF Profile and apply it in a virtual server policy. However, the Advanced Bot Protection policy does not activate until the FortiGuard ABP Application is fully analyzed and Pre-Provisioned to protect the Application.

Pre-Provisioning is required to identify all URLs that should be protected in your Application domain (such as login URLs), and the locations to which JavaScript need to be inserted to collect client information. Without these resources, FortiADC will not be able to insert the necessary JavaScript for bot detection.

Pre-Provisioning is triggered upon creating the Application, and requires 2 to 3 days to complete. During this process, your FortiGuard ABP Application will be in pending status until Pre-Provisioning is complete. When the Application status is ready, Advanced Bot Protection can be activated in your FortiADC.

When Advanced Bot Protection is initially activated, it is recommended to set the WAF action that would allow you to observe and log any events detected by the FortiGuard Advanced Bot Protection, instead of immediately setting to block.

FortiGuard Advanced Bot Protection uses a multidimensional deep learning engine to learn and track bot attacks over time by using sophisticated AI model training. As FortiGuard ABP builds its training model, it will continue to improve and refine its bot detection capabilities. However, this may mean triggering false positives in the initial stages of the AI model training.

Before you begin:
  • You must have enabled and successfully connected the Advanced Bot Protection connector on Security Fabric > Fabric Connectors.
    The Advanced Bot Protection module under Web Application Firewall is available only after the Advanced Bot Protection connector is enabled.
  • You must have Read-Write permission for Security settings.
  • You must have access to the FortiGuard Advanced Bot Protection User Portal to obtain the Application ID from an existing Application or create a new configuration. For more information, see Obtaining the Application ID from the FortiGuard ABP User Portal.

Decompression must be enabled to support JavaScript insertion, which is critical to FortiGuard Advanced Bot Protection functionality.

Ensure a Decompression policy is enabled in your HTTP/HTTPS Application Profile to allow JavaScript to be inserted into compressed HTTP/HTTPS web content. To increase performance, most websites utilize HTTP compression to reduce the size of transmitted data. This compressed HTTP/HTTPS content must be decompressed to allow FortiADC to insert the required JavaScript tag to the HTML. If the real server response is not compressed, then decompression is unnecessary. However, if the real server response is compressed then decompression must be enabled, otherwise the JavaScript will fail to insert.

To configure an Advanced Bot Protection policy:
  1. Go to Web Application Firewall > Advanced Bot Protection.
  2. In the Advanced Bot Protection tab, click Create New to display the configuration editor.
  3. Configure the following Advanced Bot Protection settings:

    Setting

    Description

    Name

    Specify a name for the Advanced Bot Protection policy.
    Valid characters are A-Z, a-z, 0-9, _, and -. No spaces. The configuration name cannot be edited once it has been saved.

    Status

    Enable/disable the status of this Advanced Bot Protection policy.

    Status must be enabled to display Advanced Bot Protection configuration options.

    Application ID

    Specify the Application ID assigned to your FortiGuard ABP Application.

    The Application ID is used to bind this Advanced Bot Protection policy to the FortiGuard ABP Application.

    For steps on how to obtain the Application ID from the FortiGuard ABP User Portal, see Obtaining the Application ID from the FortiGuard ABP User Portal.

    Action

    Specify a WAF action object to apply when a bot is detected. You can specify a predefined or user-defined WAF action profile. (See Configuring WAF Action objects.)

    Predefined WAF actions:

    • alert — WAF policies will allow the traffic to pass and log the event.
    • block — WAF policies will drop the current attack session by HTTP 403 message and block the attacker (according the attacker’s IP address) for 1 hour, and log the event.
    • captcha — WAF policies will allow the traffic to pass if the client successfully fulfills the CAPTCHA request, and log the event.
    • deny — WAF policies will the drop current attack session by HTTP 403 message, and log the event.
    • silent-deny — WAF policies will drop the current attack session by HTTP 403 message, without logging the event.

    The default action is alert.

    Severity

    Select the event severity to log when a bot is detected:

    • High — Log as high severity events.
    • Medium — Log as a medium severity events.
    • Low — Log as low severity events.

    The default is Low.

    Exception Name

    Select an exception configuration object. Exceptions identify specific hosts or URL patterns that are not subject to processing by this rule.

  4. Optionally, click Verify to validate the inputted Application ID against multiple parameters to ensure the connection between FortiADC and the FortiGuard ABP Application is successfully established.

    For more information, see Verifying the Application ID and connection status to FortiGuard ABP.
  5. Click Save.
    Once the Advanced Bot Protection policy is saved, you can reference it in a WAF Profile configuration.

It is strongly recommended to verify the Application ID and FortiGuard ABP server connection prior to completing the Advanced Bot Protection policy configuration. Even though this is an optional step, it is helpful to diagnose any potential issues and apply fixes early.

When the Advanced Bot Protection policy is created, an internal verification is automatically conducted to verify the status of the Application ID and FortiGuard ABP server connection. If the Application ID is not valid, or any other validation parameters has failed, the Advanced Bot Protection policy will fail to function and the system will log the failure to send the ABP policy.

Verifying the Application ID and connection status to FortiGuard ABP

The Advanced Bot Protection policy can only function if the Application ID is valid and the connection to the FortiGuard ABP server is successfully established. FortiADC validates multiple parameters, including if the Application ID is available, the FortiGuard ABP server connectivity, if the FortiGuard ABP license is valid. There are two types of messages as differentiated by text color: green text indicates a positive status where all required parameters are validated successfully; and red text that indicate one or more parameters did not pass validation.

The following table describes some common verification results.

Verification status message

Guidelines

Success (green)

All required parameters pass validation; application ID is available, FortiGuard ABP server certificate is valid, network connectivity is good, etc.

Application not found (red)

The Application ID does not exist. This could be an input error.

Account license invalid (red)

The FortiGuard ABP license is not valid. Please verify your license details or contact Fortinet Support.

Couldn't connect to server (red)

Unable to connect to the FortiGuard ABP server. Please check your network settings.

Couldn't resolve hostname (red)

Unable to resolve the hostname of the FortiGuard ABP server. Please check your network settings.

No available SN cert (red)

The device does not have an available SN certificate. Please check your local certificate.

No available CA cert (red)

The device does not have an available CA certificate. Please check your CA certificate.

Problem with the local certificate

An error occurred with the remote server certificate. Please check your local certificate.

SSL peer certificate or SSH remote key was not OK

An error occurred with the remote server certificate involving the SSL peer certificate or SSH remote key. Please check your local certificate.