Anomaly Detection
Use machine learning enabled Anomaly Detection to block zero day threats and other sophisticated attacks. Machine learning automatically and continuously builds and maintains a model of normal user behavior and uses it to identify malicious application traffic. To learn about whether a request is legitimate or a potential malicious attack attempt, it performs the following tasks:
- Captures and collects inputs, such as URL parameters, to build a mathematical model of allowed access
- Matches anomalies against pre-trained threat models
- Detects attacks
Once an anomaly is triggered by the mathematical model, FortiWeb Cloud uses pre-built trained threat models to confirm whether it's a real attack or just a benign anomaly that should be ignored.
Model settings
FortiWeb Cloud parses all the URLs in a domain, and builds anomaly detection models for all parameters attached to the URLs.
After anomaly detection model is built, the system will keep on calculating the probability of the new samples and compare it against the model. If the probability of the new samples varies to a large extent for a long period, the system determines this parameter has changed and automatically rebuilds the model based on the new samples.
To configure anomaly detection:
- Go to SECURITY RULES> Anomaly Detection.
You must have already enabled this module in Add Modules. See How to add or remove a module. - Configure the following settings.
- Trust: The system will collect samples only from the IP ranges in the Source IP list.
- Block: The system will collect sample from any IP addresses except the ones in the Source IP list.
- Select the action that FortiWeb Cloud takes when it detects a violation of the rule from the top right corner.
To configure the actions, you must first enable the Advanced Configuration in Global > System Settings > Settings.Alert
Accept the request and generate a log message.
Alert & Deny
Block the request (or reset the connection) and generate a log message.
- Click SAVE.
| IP List Type |
Whichever option you choose, if you leave the Source IP list blank, the system will collect traffic data samples from any IP address. |
|
Source IP List |
Click Create New to list the IP ranges of the samples. Depending on whether you select Trust or Block, FortiWeb Cloud will or will not collect samples from the specified IP ranges. |
|
Due to database migration, the Anomaly Detection machine learning data will be removed after upgrade to 23.1. The system will rebuild the model after upgrade. |
Overview
The Overview tab provides a high level summary of data collected for the domain, including Top 10 URLs by Hit, Violations triggered by anomalies, HMM learning process, Event Dashboard.
Domain overview
The top of the Overview page provides a summary of the data that the machine-learning module has learned about the domain.
| Parameters | Description |
|---|---|
| Access Frequency |
Indicates how frequently this application is being accessed.
|
| Start Time |
The date and time when the machine-learning module started to learn about the domain. |
| URL Number |
The total number of URLs that the machine-learning module has learned. |
| Block |
The total number of block actions that have been triggered since the start time up to the present moment. |
| Service(HTTP/HTTPS) |
The total amount of the HTTP and the HTTPS traffic from the start time up to now. |
| Page Charset |
The charset of URLs in the domain, such as UTF-8. |
Top 10 URLs by Hit
This chart displays the top 10 URLs for page hits counts.
Violations Triggered by Anomalies
This chart displays the total number of the potential anomalies and definite anomalies found by the anomaly detection profile.
Learning Progress
This chart displays the statistics of machine learning states of all parameters in the domain. Hover over the circle to check how many parameters are in Collecting, Building, Testing, Running, or Discarded stages respectively. For the explanation of each stage, see Anomaly Detection.
Machine Learning Events
This chart displays the anomaly detection events, such as sample collection, model running, building and testing, along with the time periods when these events take place.
Tree View
This tab displays the entire URL directory of the domain in a tree view. You can choose either one of the URLs to view its violation statistics.
Web site directory
The left panel of the Tree View page shows the directory structure of the website. The / (backslash) indicates the root of the site. You can click a URL in the directory tree, then the violation statistics of this URL will be displayed on the right side of the Tree View page. You can also click a directory, then click Relearn Directory or Rebuild Directory to relearn or rebuild anomaly detection models for all the URLs under the selected directory.
URL summary
This part of the Tree View page shows the statistics of a specific URL.
| Parameters | Description |
|---|---|
| Access Frequency |
The frequency at which this URL was accessed in last 24 hours. The frequency is divided into 7 levels, as defined below:
|
| Model Initialization Date |
The date and time when the mathematical model of this URL was initialized. It shows when FortiWeb Cloud began to learn about the data of this URL. |
| Block |
The total number of block actions that have been triggered against this URLsince the start time up to the present moment. |
|
Anomaly |
The anomalies detected by the anomaly detection model. |
Violation Trend
This chart shows the trend of violations in last 24 hours.
Parameter list
The Parameters list shows all the parameters attached to the URL. For example, if the URL is http://www.demo.com/1.php?user_name=jack, then user_name is the parameter. The system builds machine learning model for each parameter, and detects the abnormal parameter values.