Log buffer on FortiGates with an SSD disk

FortiGates with an SSD disk have a configurable log buffer. When the connection to FortiAnalyzer is unreachable, the FortiGate is able to buffer logs on disk if the memory log buffer is full. The logs queued on the disk buffer can be sent successfully once the connection to FortiAnalyzer is restored.

The queued logs are buffered to the memory first and then disk. If the total buffer is full, new logs will overwrite the old logs.

To configure the log buffer:

1. Allocate disk space (MB) to temporarily store logs to FortiAnalyzer:

   config system global
       set faz-disk-buffer-size 200
   end

2. Check the fgtlogd statistics. The 200 MB disk buffer has been set, and there are currently no logs buffered in memory or on disk when FortiAnalyzer is reachable:

   # diagnose test application fgtlogd 41 
   cache maximum: 19569745(18MB) objects: 0 used: 0(0MB) allocated: 0(0MB)
      VDOM:root
   Memory queue for: global-faz
           queue:
                   num:0 size:0(0MB) total size:0(0MB) max:15375441(14MB) logs:0
           queue disk total size:0MB, max size:200MB
                   total items:0
                           devid:-1-13-0-0
                           buffer path:/var/log/log/qbuf/13.0/0
                           saved size:0MB, lost files: 0
                           save roll:0 restore roll:0
   Confirm queue for: global-faz
           queue:
                   num:0 size:0(0MB) total size:0(0MB) max:15375441(14MB) logs:0

3. Disable the connection between the FortiGate and FortiAnalyzer. For example, delete the FortiGate from the FortiAnalyzer authorized device list.

   Assuming a massive number of logs (~ 300000) are recorded during this downtime, the logs will be queued in the memory buffer first. If the memory buffer is full, then the remaining logs will be queued on the disk buffer.

4. Recheck the fgtlogd statistics. Currently, there are logs buffered in both memory and disk:

   # diagnose test application fgtlogd 41 
   cache maximum: 19569745(18MB) objects: 14391 used: 10450754(9MB) allocated: 12089088(11MB)
      VDOM:root
   Memory queue for: global-faz
           queue:
                   num:14245 size:9306505(8MB) total size:10450754(9MB) max:15375441(14MB) logs:14245
           queue disk total size:199MB, max size:200MB
                   total items:321085
                           devid:-1-13-0-0
                           buffer path:/var/log/log/qbuf/13.0/0
                           saved size:199MB, lost files: 10
                           save roll:60 restore roll:10
   Confirm queue for: global-faz
           queue:
                   num:0 size:0(0MB) total size:10450754(9MB) max:15375441(14MB) logs:0

   The overall fgtlogd statistics shows the total cached logs is the sum of the logs buffered in memory and on disk:

   # diagnose test application fgtlogd 4
   Queues in all miglogds: cur:1  total-so-far:727973
   global log dev statistics:
   faz=399411, faz_cloud=0, fds_log=399411
   faz 0: sent=0, failed=0, cached=335480, dropped=0
   Num of REST URLs: 0

5. Enable the connection between FortiAnalyzer and the FortiGate.
6. After a while, check the fgtlogd statistics to confirm that all buffered logs are being sent to FortiAnalyzer successfully:

   # diagnose test application fgtlogd 4
   Queues in all miglogds: cur:1  total-so-far:727973
   global log dev statistics:
   faz=399411, faz_cloud=0, fds_log=399411
   faz 0: sent=335487, failed=0, cached=0, dropped=0
   Num of REST URLs: 0                      
   

   # diagnose test application fgtlogd 41
   cache maximum: 19569745(18MB) objects: 0 used: 0(0MB) allocated: 0(0MB) 
   VDOM:root
   Memory queue for: global-faz
           queue:
                   num:0 size:0(0MB) total size:0(0MB) max:15375441(14MB) logs:0
           queue disk total size:0MB, max size:200MB
                   total items:0
                           devid:-1-13-0-0
                           buffer path:/var/log/log/qbuf/13.0/0
                           saved size:0MB, lost files: 10
                           save roll:60 restore roll:60
   Confirm queue for: global-faz
           queue:
                   num:0 size:0(0MB) total size:0(0MB) max:15375441(14MB) logs:0