Threat feeds

Threat feeds dynamically import an external block list from an HTTP server in the form of a plain text file, or from a STIX/TAXII server. Block lists can be used to enforce special security requirements, such as long term policies to always block access to certain websites, or short term requirements to block access to known compromised locations. The lists are dynamically imported, so that any changes are immediately imported by FortiOS.

There are four types of threat feeds:

http://example/com.url
https://example.com/url
http://example.com:8080/url

192.168.2.100
172.200.1.4/16
172.16.1.2/24
172.16.8.1-172.16.8.100
2001:0db8::eade:27ff:fe04:9a01/120
2001:0db8::eade:27ff:fe04:aa01-2001:0db8::eade:27ff:fe04:ab01

mail.*.example.com
*-special.example.com
www.*example.com
example.com

External resources file format

File format requirements for a HTTP/HTTPS external resources file:

• The file is in plain text format with each URL list, IP address, domain name, or malware hash occupying one line.
• The file is limited to 10 MB or 128 × 1024 (131072) entries, whichever limit is hit first.
• The entry limit also follows the table size limitation defined by CMDB per model.
• The external resources update period can be set to 1 minute, hourly, daily, weekly, or monthly (43200 min, 30 days).
• The external resources type as category (URL list) and domain (domain name list) share the category number range 192 to 221 (total of 30 categories).
• There is no duplicated entry validation for the external resources file (entry inside each file or inside different files).
• If the number of entries exceed the limit, a warning is displayed. Additional entries beyond the threshold will not be loaded.

For domain name list (type = domain):

• Simple wildcards are allowed in the domain name list, for example: *.test.com.
• IDN (international domain name) is supported.

For IP address list (type = address):

• The IP address can be a single IP address, subnet address, or address range. For example, 192.168.1.1, 192.168.10.0/24, or 192.168.100.1-192.168.100.254.
• The address can be an IPv4 or IPv6 address. An IPv6 address does not need to be in [ ] format.

For URL list (type=category):

• The scheme is optional, and will be truncated if found; https:// and http:// are not required.

• Wildcards are allowed at the beginning or end or the URL, for example: *.domain.com or domain.com.*.

• IDN and UTF encoding URL are supported .

• The URL can be an IPv4 or IPv6 address. An IPv6 URL must be in [ ] format.

To determine the external resource table size limit for your device:

# print tablesize
...
system.external-resource: 0 256 512
...

For this device, a FortiGate 60E, the global limit is 512 and the limit per VDOM is 256.

Configuring a threat feed

When configuring a threat feed, there are two options available for the update method, external feed or push API.

When multi VDOM mode is enabled, threat feed external connectors can be defined in the global VDOM or within a VDOM. See Threat feed connectors per VDOM for example configurations.

Configuring a threat feed with an external feed update

The threat feed will periodically fetch entries from the URI using HTTP or HTTPS.

To configure the threat feed in the GUI:

1. Go Security Fabric > External Connectors and click Create New.
2. In the Thread Feeds section, select the required feed type.
3. Configure the connector settings:

   Status	Enable/disable the connector.
   Name	Enter a name for the threat feed connector.
   Update method	Select External Feed.
   URI of external resource	Enter the link to the external resource file. HTTP, HTTPS, and STIX protocols are supported.
   HTTP basic authentication	Enable/disable basic HTTP authentication. When enabled, enter the username and password in the requisite fields.
   Refresh Rate	The time interval to refresh the external resource, in minutes (1 - 43200, default = 5). The applicable threat feed will be triggered to refresh between 0 minutes and the configured value. When the refresh is triggered, if another task is being processed be the schedule worker, the refresh task will be added to the queue.
   Comments	Optionally, enter a description of the connector.

4. Click OK.

To configure the threat feed in the CLI:

config system external-resource
    edit <name>
        set status {enable | disable}
        set type {category | address | domain | malware}
        set category <integer>
        set username <string>
        set password <string>
        set comments <string>
        *set resource <resource-uri>
        set user-agent <string>
        *set refresh-rate <integer>
        set source-ip <ip address>
        set interface-select-method {auto | sdwan | specify}
    next
end

Parameters marked with an asterisk (*) are mandatory and must be filled in. Other parameters either have default values or are optional.

Configuring a threat feed with a push API update

The threat feed receives entry updates from webhook requests to the FortiGate REST API. This method provides the code samples needed to perform add, remove, and snapshot operations.

In the following example, a FortiGuard Category threat feed is used to show the different API push options.

To configure the threat feed in the GUI:

1. Go to Security Fabric > External Connectors and click Create New.

2. In the Threat Feeds section, click FortiGuard Category.

3. Enter a name.

4. Set the Update method to Push API.

5. Click OK. The Threat Feed Push API Information pane opens that contains the following fields:

   • URL: the FortiGate's API URL to call in order to perform the update.

   • API admin key: when an API administrator user is configured on the FortiGate, an API admin key will be associated with the API administrator. Input the API key to see the final cURL request.

   • Push command: select one of three push methods.

     • Add: add the specified entries to the threat feed.

       

     • Remove: remove the specified entries from the threat feed.

       

     • Snapshot: replace the threat feed with all specified entries.

       

   • Entries: enter the entries separated by a comma (,) to be applied to the FortiGuard Category threat feed list.

   • Sample cURL request: copy this cURL command to perform the push API update on the FortiGate against the list (cccccccc).

   See REST API administrator for more information.

6. Copy the content in the Sample cURL request field (Add is used in this example).

7. Click OK.

8. On a client, generate the API request for the threat feed.

To configure the threat feed in the CLI:

config system external-resource
    edit "cccccccc"
        set update-method push
        set category 201
    next
end

To use the API in the CLI:

# diagnose system external-resource {push-add | push-remove | push-snapshot} <feed_name> <entry>

To use the API with a JSON file:

# diagnose sys external-resource push-api-json-commands

{
  "commands": [<array (mandatory)>
    {<object (mandatory)>
         "name": <string (mandatory)>,
      "command": <string (mandatory, "add", "remove", or "snapshot")>,
      "entries": [<array (mandatory)>
        <string (mandatory, such as "10.100.1.1")>,
      ]
    }
  ]
}

Sample:

# diagnose sys external-resource push-api-json-commands '{"commands":[{"name":"test","command":"add","entries":["10.10.10.1","10.10.10.2"]},{"name":"test","command":"whatever","entries":["10.10.10.3","10.10.10.4"]}]}'
command returned: EXT_RESOURCE_PUSH_CMD_RETURN_OK
Returned json:
[
  {
    "name":"test",
    "command":"add",
    "status":"success"
  },
  {
    "name":"test",
    "command":"whatever",
    "error":"Invalid command.",
    "status":"error"
  }
]

To use the API with a Postman REST client:

1. Create an API administrator in FortiOS with write access.

2. Ensure the API token is generated.

3. Configure the external resource list as needed.

4. In the Postman client, create a new request, set the HTTP method to POST, enter the URL.

5. Configure the access token using one of the following methods:

   • To use the bearer token: click the Authorization tab, set the Type to Bearer, and enter the REST API administrator token.

   • To use the access_token parameter: click the Params tab and enter the access_token key-value pair (access_token and <key>).

6. Click the Body tab and configure the following:

   1. Select raw and set the input type to JSON.

   2. Insert the JSON data payload.

7. Click Send to send the POST request. If there is a response, the response body appears. For example,

   POST https://172.18.52.153/api/v2/monitor/system/external-resource/dynamic?access_token=g1mnfs8bzxk5hf8Qwcz4kx7yn3jHmG&vdom=vd1
   Content-Type: application/json
   User-Agent: PostmanRuntime/7.29.2
   Accept: */*
   Postman-Token: 04e10736-190e-4119-92e1-04e91bf99c10
   Host: 172.18.52.153
   Accept-Encoding: gzip, deflate, br
   Connection: keep-alive
   Content-Length: 485
   
   {
      "commands":[
         {
            "name":"ip",
            "command":"add",
            "entries":[
               "10.10.10.1",
               "10.10.10.2"
            ]
         },
         {
            "name":"fqdn",
            "command":"remove",
            "entries":[
               "10.10.10.1",
               "10.10.10.2"
            ]
         },
         {
            "name":"fortiguard",
            "command":"snapshot",
            "entries":[
               "10.10.10.1",
               "10.10.10.2"
            ]
         }
      ]
   }
   
   HTTP/1.1 200 OK
   date: Fri, 22 Jul 2022 21:10:39 GMT
   x-frame-options: SAMEORIGIN
   content-security-policy: frame-ancestors 'self'
   x-xss-protection: 1; mode=block
   cache-control: no-cache, must-revalidate
   content-length: 480
   content-type: application/json
   Connection: keep-alive
   
   {
      "http_method":"POST",
      "results":[
         {
            "name":"ip",
            "command":"add",
            "status":"success"
         },
         {
            "name":"fqdn",
            "command":"remove",
            "status":"success"
         },
         {
            "name":"fortiguard",
            "command":"snapshot",
            "status":"success"
         }
      ],
      "vdom":"vd1",
      "path":"system",
      "name":"external-resource",
      "action":"dynamic",
      "status":"success",
      "serial":"FG6H1E5819900000",
      "version":"v7.2.1",
      "build":1254
   }

Viewing the update history

To review the update history of a threat feed, go to Security Fabric > External Connectors, select a feed, and click Edit. The Last Update field shows the date and time that the feed was last updated.

Click View Entries to view the current entries in the list.

EMS threat feed

A FortiGate can pull malware threat feeds from FortiClient EMS, which in turn receives malware hashes detected by FortiClients. The malware hash can be used in an antivirus profile when AV scanning is enabled with block or monitor actions. See Malware threat feed from EMS for an example.