External feeds

The FortiGate dynamically imports an external list from an HTTP/HTTPS server in the form of a plain text file. The imported list is then available as an external feed, which can be used to enforce special security requirements, such as long-term policies to always allow or block access to certain websites, or short-term requirements to block access to known compromised locations. The external feeds are dynamically synchronized and are updated periodically so that any changes are immediately imported by FortiOS.

If the FortiGate loses connectivity with the external server, the external feed will continue to function despite the Connection Status error or reboot. However, the external feed will not be updated and no new entries will be added until the connection is re-established.

FortiOS also supports STIX/TAXII format. See STIX format for external feeds for more information.

There are five types of external feeds:

FortiGuard Category	The FortiGate dynamically imports a text file from an external server, which contains one URL per line. See FortiGuard category external feed for more information.
IP Address	The FortiGate dynamically imports a text file from an external server, which contains one IP/IP range/subnet per line. See IP address external feed for more information.
Domain Name	The FortiGate dynamically imports a text file from an external server, which contains one domain per line. Simple wildcards are supported. See Domain name external feed for more information.
MAC Address	The FortiGate dynamically imports a text file from an external server, which contains one MAC address, MAC range, or MAC OUI per line. See MAC address external feed for more information.
Malware Hash	The FortiGate dynamically imports a text file from an external server, which contains one hash per line in the format <hex hash> [optional hash description]. Each line supports MD5, SHA1, and SHA256 hex hashes. See Malware hash external feed for more information.

Additionally, the EMS external feed is integrated with FortiClient EMS, but it is not configured in the same way as the preceding feeds:

EMS External Feed

A FortiGate can pull malware external feeds from FortiClient EMS, which in turn receives malware hashes detected by FortiClient. The malware hash can be used in an antivirus profile when AV scanning is enabled with block or monitor actions. See Malware external feed from EMS for an example.

FortiManager can host external feeds. See External resources in the FortiManager Administration Guide.

External resources file format

File format requirements for a HTTP/HTTPS external resources file:

• The file is in plain text format with each URL list, IP address, domain name, or malware hash occupying one line.

  Comments can be added by using the number sign, for example: # This is a test.

• The file is limited to a maximum size and entry limit, based on the device model; see External resource entry limit.

• The external resources update period can be set to 1 minute, hourly, daily, weekly, or monthly (43200 min, 30 days).

• The external resources type as category (URL list) and domain (domain name list) share the category number range 192 to 221 (total of 30 categories).

• There is no duplicated entry validation for the external resources file (entry inside each file or inside different files).

• If the number of entries exceed the limit, a warning is displayed. Additional entries beyond the threshold will not be loaded.

For URL list (type = category):

• The scheme is optional, and will be truncated if found; https:// and http:// are not required.

• Wildcards are allowed at the beginning or end or the URL, for example: *.domain.com or domain.com.*.

• IDN and UTF encoding URL are supported .

• The URL can be an IPv4 or IPv6 address. An IPv6 URL must be in [ ] format.

• If using an IP address, it cannot be a private IP address.

For IP address list (type = address):

• The IP address can be a single IP address, subnet address, or address range. For example, 192.168.1.1, 192.168.10.0/24, or 192.168.100.1-192.168.100.254.

• The address can be an IPv4 or IPv6 address. An IPv6 address does not need to be in [ ] format.

For domain name list (type = domain):

• Simple wildcards are allowed in the domain name list, for example: *.test.com.

• IDN (international domain name) is supported.

For MAC address list (type = mac-address):

• The MAC address can be a single MAC address, MAC OUI, or MAC range. For example, 01:01:01:01:01:01, 8c:aa:b5, or 01:01:01:01:01:01-01:01:02:50:20:ff.

• The hexadecimal digits in MAC address must only be separated by colons.

For malware hash list (type = malware):

• The malware hash list follows a strict format in order for its contents to be valid. Malware hash signature entries must be separated into each line. A valid signature must follow this format:

  # MD5 Entry with hash description
  aa67243f746e5d76f68ec809355ec234  md5_sample1
  
  # SHA1 Entry with hash description
  a57983cb39e25ab80d7d3dc05695dd0ee0e49766  sha1_sample2
  
  # SHA256 Entry with hash description
  ae9bc0b4c5639d977d720e4271da06b50f7c60d1e2070e9c75cc59ab30e49379  sha256_sample1
  
  # Entry without hash description
  0289b0d967cb7b1fb1451339c7b9818a621903090e0020366ab415c549212521
  
  # Invalid entries
  7688499dc71b932feb126347289c0b8a_md5_sample2
  7614e98badca10b5e2d08f8664c519b7a906fbd5180ea5d04a82fce9796a4b87sha256_sample3

To determine the external resource table size limit for your device:

# print tablesize
...
system.external-resource: 0 256 512 0
...

In this example, a FortiGate 60E has a global limit of 512 and a per-VDOM limit of 256. A FortiGate 60E can configure up to 512 feeds. The total number of feeds is limited by the available memory on the device.

External resource entry limit

The external resource entry limit is global, and file size restrictions change according to the device model. If VDOMs are enabled, global entries are counted first, then VDOM entries in alphabetical order based on the VDOMs' names.

If more than the maximum number of entries are added, the most recently added entries are truncated unless the order is manually changed. The entry order can be changed using the move CLI command. For example:

config system external-resource
    move "entry2" before "entry1"
end

The maximum number of each type of entry and the file size limit for each model range are as follows:

	High-End (Data Center)	Mid-Range (Campus)	Entry-Level (Branch)
Category	2 000 000	300 000	150 000
IP address	300 000	300 000	300 000
Domain	5 000 000	3 000 000	1 000 000
MAC	1 000 000	1 000 000	1 000 000
File size limit (MB)	128	64	32

For example, a FortiGate 601E, a mid-range device, is configured as follows:

• global VDOM: One external feed, g-category-push, with one entry.

• root VDOM: One external feed, r-category-push, with one entry.

• vd1 VDOM: Two external feeds, v‑category‑300000 with 300000 entries first, and v‑category‑push with one entry second.

• vd2 VDOM: One external feed, z-category-push, with one entry.

There are more than 300000 entries, so some of the entries will be truncated.

• The global VDOM is counted first, so its entry is kept:

  FGT (global)# diagnose sys external-resource stats
  name: g-category-push; uuid_idx: 606; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: no; buildable: 1; total in count file: 1;

• The root VDOM is alphabetically before the vd1 and vd2 VDOMs, so its entry is kept:

  FGT (root)# diagnose sys external-resource stats
  name: g-category-push; uuid_idx: 606; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: no; buildable: 1; total in count file: 1;
  name: r-category-push; uuid_idx: 746; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: no; buildable: 1; total in count file: 1;

• The vd1 VDOM is next alphabetically. The maximum number of entries is 300000, so 299998 entries from the v‑category‑3000000 external feed are kept, and no entries from the v‑category‑push feed:

  FGT (vd1)# diagnose sys external-resource stats
  name: g-category-push; uuid_idx: 606; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: no; buildable: 1; total in count file: 1;
  name: v-category-300000; uuid_idx: 863; type: category; update_method: feed; truncated total lines: 300000; valid lines: 299999; error lines: 1; used: no; buildable: 299998; total in count file: 300000;
  name: v-category-push; uuid_idx: 868; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: yes; buildable: 0; total in count file: 1;

• The vd2 VDOM is last alphabetically and the maximum number of entries has already been reached, so all of its entries are truncated:

  FGT (vd2)# diagnose sys external-resource stats
  name: g-category-push; uuid_idx: 606; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: no; buildable: 1; total in count file: 1;
  name: z-category-push; uuid_idx: 989; type: category; update_method: push; total lines: 1; valid lines: 1; error lines: 0; used: no; buildable: 0; total in count file: 1;