Configuring ML Based API Protection policy

The machine learning based API Protection learns the REST API data structure from user traffic samples and then build a mathematical model to screen out malicious API requests.

It analyzes the method, URL, and endpoint data of the API request samples to generate an API data structure file for your application. This model describes the API data schema model of endpoint data. If the incoming API request violates the data structure, it will be detected as an attack.

API Protection supports JSON request body.

Continuous learning in ML based API Protection

Starting from version 7.4.0, ML-based API protection includes the feature of continuous adjustment for its API learning models. This allows the models to adapt to changes in the API schema, including the introduction of new APIs and modifications to existing parameters. To enable this feature, set the Action to Standby.

It's important to note that continuous learning mode is designed to generate an API schema file and does not support blocking malformed API requests.

To utilize the schema file for screening out malformed requests, you can download the API schema file in API View tab then upload it to API Validation feature. By doing so, the system will be able to compare incoming API requests against the up-to-date API schema, effectively identifying and filtering out any malformed requests. This combination of continuous learning mode and API Validation allows for more robust protection against malformed API requests while keeping the learning models up-to-date with the evolving API schema.

ML based API Protection is only supported in standalone and HA active-passive modes, and it's only supported in Reverse Proxy mode.

To create an API Protection policy:

API Protection policy is part of a server policy. It is created on the Policy > Sever Policy page.

1. Click Policy > Server Policy.
2. Select an existing server policy.
   Please note that the API Protection Machine Learning policies can't be created during the server policy creation process. You should first create a server policy, then click Edit to create a API Protection Machine Learning policy.
3. Scroll down to the Machine Learning section at the bottom of the page, click the API Protection tab, then click Create. The New Machine Learning dialog opens.
   
4. Click the + (Add) sign after the Domain filed to add the desired domains, so that the system collects samples and builds up a API Protection Machine Learning model for the domains.
5. Select whether to trust or block the specified source IP addresses.
6. Click the + (Add) sign after the IP Range field to add IP/Range, so as to limit the system to collect data only (When IP List Type is Trust) or exclude data (When IP List Type is Block) from the specified IP range.
   
7. Click OK.

After it's completed, go back to Server Policy. Select the one which contains the API Protection policy you just created. You will see the following buttons in the API Protection tab.

Button	Function
View	Click to view and edit API Protection policies and their learning results. Note: You can also access the API Protection page by clicking API Protection > ML Based API Protection, and then selecting a specific policy.
Start/Stop	Click to start/stop API Protection machine learning for the policy.
Refresh	Click to restart API Protection model building for all the domains in the policy. Note: This will discard all existing learning results and then relearn all data.
Discard	Click to remove all learned data from the policy.
Export	Click to export the data for all the domains, including the model data and configurations.
Import	Click to import the API Protection data from your local directory to FortiWeb. Note: The API Protection model generated in FortiWeb 7.0 cannot be imported in FortiWeb 7.0.1, and vice versa.

All API Protection policies that you have created are displayed on the API Protection > ML Based API Protection page, where you can edit them to your preference.

To configure an API Protection policy:

1. Click API Protection > ML Based API Protection .
2. Double-click the server policy that contains the desired API Protection policy (or highlight it and then click the Edit button on top of the page) to open it. The Edit API Protection Configuration page opens, which breaks down API Protection policy into several sections, each of which has various parameters you can use to configure the policy.
3. In Domain List, add domains to be protected by the API Protection Policy.
   1. Click Create New. The Edit domain settings page will open.
   2. Enter the host address. You can enter the exact string or use wildcard to match multiple domains.
   3. The system by default learns API requests to all the URL paths of the domain. If you want to restrict the learning to certain API paths, enable Restrict Learning Path, then perform the following steps to specify the API paths to be learned.
      1. Click Create New.
      2. For URL Type, select whether the API pattern must contain a literal URL (Simple String), or a regular expression designed to match multiple URLs (Regular Expression).
      3. For URL Expression, type either:
         • The literal URL, such as /folder1/index.htm that the HTTP request must contain in order to match the rule, or use wildcards to match multiple URLs, such as /folder1/* or /folder1/*/index.htm. The URL must begin with a slash ( / ).
         • A regular expression, such as ^/*\.jsp\?uid\=(.*), matching all and only the URLs to which the rule should apply. The pattern does not require a slash ( / ); however, it must at least match URLs that begin with a slash, such as /profile.cfm.

           Do not include the domain name, such as www.example.com, which is configured separately in the Host drop-down list.

           To create and test a regular expression, click the >> (test) icon. This opens the Regular Expression Validator window where you can fine-tune the expression. For details, see Regular expression syntax and Cookbook regular expressions.

   4. Click OK on Add Restricted API Learning Path page.
   5. Click OK on the Edit domain settings page.

   The system will start building API Protection model when 100 API request samples are collected for the specified domain. You can change the sample count through set start-training-cnt <int> in config waf api-learning-policy.

   Once the domains are added, they will be shown under the Domain List section. You can click at the right corner of the section to choose whether to show the domains in Grid View or List View.

4. In Action Settings, Configure the action that FortiWeb will take when it detects malicious API requests. The following settings apply to all the API paths in your domain. If you want to change the action setting for a specific API, see Editing and viewing machine learning models for API paths

   The API protection module provides two layers of protections:

   • Threat Protection: It learns parameter value patterns from API body requests and builds mathematical models to screen out abnormal requests that are deemed malicious.

   • Schema Protection: It analyzes the method, URL, and endpoint data of the API request samples to generate an API data structure file for your application. If the incoming API request violates the data structure, it will be detected as an attack.

   Action	Select which action FortiWeb will take when it detects an API violation: Alert—Accept the connection and generate an alert email and/or log message. Alert & Deny—Block the request (or reset the connection) and generate an alert and/or log message. Period Block—Block subsequent requests from the client for a number of seconds. Also configure Period Block. Standby— Available only for Schema Protection. Selecting Standby mode will activate the continuous learning mode. The system will continuously adjust the API learning models to adapt to changes in the API schema. This includes scenarios such as the introduction of new APIs, modifications to existing parameters, etc. It is important to note that blocking violations is not supported in continuous learning mode at present. However, you can go to API View to download the learned schema, then upload it to API Validation, which allows you to block malformed API requests. Disable— Disable Threat Protection.
   Block Period	Enter the number of seconds that you want to block the requests. The valid range is 1–3,600 seconds (1 hour). This option only takes effect when you choose Block Period in Action.
   Severity	Select the severity level for this anomaly type. The severity level will be displayed in the alert email and/or log message.
   Trigger Action	Select a trigger policy that you have set in Log&Report > Log Policy > Trigger Policy. If potential or definite anomaly or HTTP Method Violation is detected, it will trigger the system to send email and/or log messages according to the trigger policy.

5. Enable Advanced Settings to proceed to step 7 and 8.
6. Add IP ranges in the Source IP list, then select Trust or Black to allow or disallow collecting traffic data samples from these IP addresses.
   1. Trust: The system will collect samples only from the IP ranges in the Source IP list.
   2. Block: The system will collect samples from any IP addresses except the ones in the Source IP list

   Whether selecting Trust or Black, if you leave the Source IP list blank, the system will collect traffic data samples from any IP addresses.

7. Select the name of the URL Replacer Policy that you have created in Machine Learning Templates.

   If web applications have dynamic URLs or unusual parameter styles, you must adapt URL Replacer Policy to recognize them.

   If you have not created an URL Replacer Policy yet, you can leave this option empty for now, and then edit this policy later when the URL Replacer Policy is created. For more information on URL Replacer Policy, see Configure a URL replacer rule

8. Click OK when done.

The system collects samples for the specified domains and analyzes the parameter, body, and the response structure of API requests to all the API paths in the domain. For how to view the machine learning model for each API path, see Editing and viewing machine learning models for API paths