Per-device log receiving rate limit

This feature adds the ability to set log rate limit per device to pause log insertion when the configured limit is exceeded. This is to prevent misconfigured devices from flooding FortiAnalyzer with unwanted data.

To configure per-device log receiving rates:

1. Go to the FortiAnalyzer CLI and use the following commands to see that log receiving rate limits are not currently set:

   FAZ3000F # config system log ratelimit

   (ratelimit)# get

   mode : disable

   (ratelimit)#

   Enter the following command to view the current logging rates for each device:

   AZ3000F # diagnose test application fortilogd 17

   # device 1_minute 10_minute rate-limit dropped

   ---------------------------------------------------------------------

   1 FGT60E9982377487 34906.10 11800.36 0 0

   2 FG800D3915800008 1988.57 690.44 0 0

   3 FGHA000947503766_CID 0.00 0.00 0 0

   4 FGT51E3U16002689 0.47 0.17 0 0

   5 FWF61FTK19000247 45387.15 15514.16 0 0

   6 FAC-VM0000000000 0.00 0.00 0 0

   7 SYSLOG-AC105101 0.00 0.00 0 0

   8 FG140P3G13800040 12895.43 4495.17 0 0

   9 FGT60E9982377480 34906.83 11805.59 0 0

   10 FG280P4614800414 0.07 0.01 0 0

   11 FG101FTK19006708 9785.22 3177.97 0 0

   12 FL-3KF3R16000142 0.00 0.00 0 0

   13 FL3K5F3M15000004 0.02 0.01 0 0

   14 FGT40FTK20025663 19553.53 7087.56 0 0

   15 System 159423.38 54571.44 0 0

2. In the CLI, use the following commands to set the log receiving rate limit to manual and configure a new default rate limit:

   FAZ3000F # config system log ratelimit

   (ratelimit)# show

   config system log ratelimit

   set mode manual

   set device-ratelimit-default 1000

   end

   Enter the following command to view the updated default rate limit applied to logging devices:

   FAZ3000F # diagnose test application fortilogd 17

   # device 1_minute 10_minute rate-limit dropped

   ---------------------------------------------------------------------

   1 FGT60E9982377487 1000.18 20781.08 1000 6680050

   2 FG800D3915800008 1000.82 1385.01 1000 210858

   3 FGHA000947503766_CID 0.00 0.00 1000 0

   4 FGT51E3U16002689 0.45 0.42 1000 0

   5 FWF61FTK19000247 1000.48 27342.83 1000 6779154

   6 FAC-VM0000000000 0.00 0.00 1000 0

   7 SYSLOG-AC105101 0.00 0.00 1000 0

   8 FG140P3G13800040 1001.70 7753.74 1000 484191

   9 FGT60E9982377480 1000.03 20778.19 1000 6680607

   10 FG280P4614800414 0.07 0.05 1000 0

   11 FG101FTK19006708 1000.52 5558.64 1000 1253065

   12 FL-3KF3R16000142 0.00 0.00 1000 0

   13 FL3K5F3M15000004 0.03 0.02 1000 0

   14 FGT40FTK20025663 1001.62 12567.55 1000 3140384

   15 System 7005.90 96167.54 42000 25228309

3. In the CLI, use the following commands to configure a per-device log rate limit for your devices.
   In this example, the FortiGate device is configured with a 2000 rate limit and FortiWiFi 61F devices are configured with a 1500 rate limit using wildcard support.

   FAZ3000F # config system log ratelimit

   (ratelimit)# show

   config system log ratelimit

   set mode manual

   config device

   edit 1

   set device "FGT60E9982377480"

   set ratelimit 2000

   next

   edit 2

   set device "FWF61F*"

   set ratelimit 1500

   next

   end

   set device-ratelimit-default 1000

   end

   Enter the following command to view the updated log rate limits:

   FAZ3000F # diagnose test application fortilogd 17

   # device 1_minute 10_minute rate-limit dropped

   ------------------------------------------------------------------

   1 FGT60E9982377487 1000.12 1000.18 1000 128460813

   2 FG800D3915800008 812.20 882.80 1000 3990480

   3 FGT51E3U16002689 0.40 0.58 1000 0

   4 FWF61FTK19000247 1501.23 1369.88 1500 48970326

   5 FAC-VM0000000000 0.00 0.00 1000 0

   6 SYSLOG-AC105101 0.00 0.00 1000 0

   7 FG140P3G13800040 0.00 704.49 1000 2673191

   8 FGT60E9982377480 2000.07 1100.19 2000 128984024

   9 FG280P4614800414 0.07 0.06 1000 0

   10 FG101FTK19006708 1000.92 914.34 1000 24125577

   11 FL-3KF3R16000142 0.00 0.00 1000 0

   12 FL3K5F3M15000004 0.02 0.02 1000 0

   13 FGT40FTK20025663 1001.30 917.09 1000 34486937

   14 System 7316.32 6889.65 42000 371691348

4. Check the alert messages in widget Alert Message console to view messages about when log rate limits are exceed and logs are dropped:

   Time Message

   Feb 25, 09:36:08 Device FGT60E0000000299 logs dropped due to exceed configured rate-limit 60 logs/sec.

   Feb 25, 09:36:08 Device FGT60E0000000435 logs dropped due to exceed configured rate-limit 60 logs/sec.

   Feb 25, 09:36:08 Device FGT60E0000000260 logs dropped due to exceed configured rate-limit 60 logs/sec.

   Feb 25, 09:36:07 Device FGT40FTK20025663 log-rate limited due to exceed configured rate-limit 1000 logs/sec.

   Feb 25, 09:36:07 Device FG101FTK19006708 log-rate limited due to exceed configured rate-limit 1000 logs/sec.

   This information is also available in local event logs:

   id=6933257507120873474 itime=2021-02-25 09:40:08 euid=1 epid=1 dsteuid=1 dstepid=1 log_id=0030039002 subtype=logging type=event level=alert time=09:40:08 date=2021-02-25 action=alert msg=Device FGT60E0000000121 logs dropped due to exceed configured rate-limit 60 logs/sec. desc=Log rate limit alert devid=FL-3KF3R16000142 devname=FL-3KF3R16000142 dtime=2021-02-25 09:40:08 itime_t=1614274808

   id=6933257502825906182 itime=2021-02-25 09:40:07 euid=1 epid=1 dsteuid=1 dstepid=1 log_id=0030039002 subtype=logging type=event level=alert time=09:40:07 date=2021-02-25 action=alert msg=Device FWF61FTK19000247 log-rate limited due to exceed configured rate-limit 1500 logs/sec. desc=Log rate limit alert devid=FL-3KF3R16000142 devname=FL-3KF3R16000142 dtime=2021-02-25 09:40:07 itime_t=1614274807