Improve FortiAnalyzer log caching

Reliable logging to FortiAnalyzer is improved to prevent lost logs when the connection between FortiOS and FortiAnalyzer is disrupted. When reliable mode is enabled:

1. Logs are cached in a FortiOS memory queue.
2. FortiOS sends logs to FortiAnalyzer, and FortiAnalyzer uses seq_no to track received logs.
3. After FortiOS sends logs to FortiAnalyzer, logs are moved to a confirm queue in FortiOS.
4. FortiOS periodically queries FortiAnalyzer for the latest seq_no of the last log received, and clears logs from the confirm queue up to the seq_no.
5. If the connection between FortiOS and FortiAnalyzer is disrupted, FortiOS resends the logs in the confirm queue to FortiAnalyzer when the connection is reestablished.

FortiAnalyzer 7.2.0 and later is required.

To enable reliable mode:

config log fortianalyzer setting
    set reliable enable
end

To view the memory and confirm queues:

1. Verify that log synchronization is enabled for FortiAnalyzer:

   # diagnose test application fgtlogd 1
   vdom-admin=0
   mgmt=root
   
   fortilog:
   faz: global , enabled 
       server=172.16.200.251, realtime=1, ssl=1, state=connected
       server_log_status=Log is allowed.,
       src=, mgmt_name=FGh_Log_root_172.16.200.251, reliable=1, sni_prefix_type=none,
       required_entitlement=none, region=ca-west-1,,
       logsync_enabled:1, logsync_conn_id:65535, seq_no:790
   ...

2. When a network disruption disconnects FortiOS from FortiAnalyzer and FortiOS continues to generate logs, the logs are cached in the memory queue.

   • View the number of logs in the cache and queue:

     # diagnose test application fgtlogd 41
     
     cache maximum: 189516595(180MB) objects: 40 used: 27051(0MB) allocated: 29568(0MB)
     
     VDOM:root
     Memory queue for: global-faz
         queue:
             num:9 size:6976(0MB) total size:26068(0MB) max:189516595(180MB) logs:28
     Confirm queue for: global-faz
         queue:
             num:29 size:19092(0MB) total size:27051(0MB) max:189516595(180MB) logs:7

     # diagnose test application fgtlogd 30
     VDOM:root
     Memory queue for: global-faz
             queue:
                     num:9 size:6976(0MB) total size:26068(0MB) max:189516595(180MB)
                             type:3, cat=1, log_count=1, seq_no=0, data len=359 size:435
                             type:3, cat=1, log_count=1, seq_no=0, data len=307 size:383
                             ......
                             type:3, cat=0, log_count=4, seq_no=0, data len=1347 size:1423
                             type:3, cat=4, log_count=1, seq_no=0, data len=653 size:729
                     'total log count':28,  'total data len':6292
     
     Confirm queue for: global-faz
             queue:
                     num:29 size:19092(0MB) total size:26068(0MB) max:189516595(180MB)
                             type:3, cat=1, log_count=1, seq_no=1, data len=290 size:366
                             type:3, cat=1, log_count=1, seq_no=2, data len=233 size:309
                             ......
                             type:3, cat=0, log_count=1, seq_no=28, data len=524 size:600
                             type:3, cat=1, log_count=1, seq_no=29, data len=307 size:383
                     'total log count':76,  'total data len':16888

     There are nine OFTP items cached to the memory queue, and 29 OFTP items to send from FortiOS to FortiAnalyzer that are waiting for confirmation from FortiAnalyzer.

   • Go to Log & Report > Log Settings to view the queue in the GUI:

     

3. Re-establish the connection between FortiOS and FortiAnalyzer and confirm that the queue has cleared by checking the seq_no, which indicates the latest confirmation log from FortiAnalyzer:

   # diagnose test application fgtlogd 30
   VDOM:root
   Memory queue for: global-faz
       queue:
           num:0 size:0(0MB) total size:0(0MB) max:189516595(180MB)
           'total log count':0,  'total data len':0
   
   Confirm queue for: global-faz
       queue:
           num:0 size:0(0MB) total size:0(0MB) max:189516595(180MB)
           'total log count':0,  'total data len':0 

   The queue has been cleared, meaning that FortiOS received confirmation from FortiAnalyzer and cleared the confirm queue.

   # diagnose test application fgtlogd 1
   vdom-admin=0
   mgmt=root
   
   fortilog:
   faz: global , enabled 
           server=172.16.200.251, realtime=1, ssl=1, state=connected
           server_log_status=Log is allowed.,
           src=, mgmt_name=FGh_Log_root_172.16.200.251, reliable=1, sni_prefix_type=none,
           required_entitlement=none, region=ca-west-1,
           logsync_enabled:1, logsync_conn_id:65535, seq_no:67
               status: ver=6, used_disk=0, total_disk=0, global=0, vfid=0 conn_verified=Y
               SNs: last sn update:38 seconds ago.
                   Sn list:
                  (FAZ-VMTM21000000,age=38s)
               queue: qlen=0.

   OFTP items with a seq_no lower than 67 have been sent to FortiAnalyzer and were confirmed.