Logs cannot be displayed on FortiAnalyzer
Besides being restored in local disk, Attack/Traffic/Event logs can also be delivered to FortiAnalyzer. This section provides troubleshooting methods when Attack/Traffic/Event logs failed to be displayed on FortiAnalyzer (abbreviated as FortiAnalyzer in below section).
The possible causes usually include:
-
FortiAnalyzer certificate issue
-
TCP connection issue with FortiAnalyzer
FortiAnalyzer certificate issue
Certificates 'fortinet-subca2001' and 'fortinet-ca2' are necessary on FortiAnalyzer for establishing SSL connection with FortiWeb. If these certs are lost on FortiAnalyzer, FortiWeb will fail to establish connection with FortiAnalyzer and thus fail to send logs to FortiAnalyzer.
-
Basic check
Check if there are 2 certificates 'Fortinet_SUBCA’ & ‘Fortinet_CA' on the FortiAnalyzer (System Settings > Certificates > CA Certificates).
If they are not there, download these two certificates from another FortiAnalyzer and import them to the current FortiAnalyzer.
-
Use diagnose commands to check and analyze certificate issues.
On FortiWeb
diagnose debug application oftp 7
diagnose debug enable
The following errors indicates failing to establish SSL connection between FortiWeb and FortiAnalyzer:
[OFTP][DEBUG](oftp_async.c:386): oftp_auth_send: auth send done fd=14...
[OFTP][DEBUG](oftp_async.c:420): oftp_auth_recv: fd=14, buf_pos=0,buf_len=12
[OFTP][DEBUG](oftp_async.c:429): oftp_auth_recv: read again : errno=Resource temporarily unavailable
On FortiAnalyzer
# diagnose debug application oftpd 8
# diagnose debug enable
The following message indicates FortiAnalyzer certificate verification failed because the necessary CA cert (CN=fortinet-ca2) is not available on the FortiAnalyzer.
FortiWeb sends its cert (CN = FortiWeb) to FortiAnalyzer for auth. This cert is signed by an intermediate CA (fortinet-subca2001) and the root CA (fortinet-ca2). FortiAnalyzer needs the 2 CA certs to verify the received cert.
[__verify_callback:475] VERIFY ERROR: depth=2, error=self signed certificate in certificate chain: /C=US/ST=California/L=Sunnyvale/O=Fortinet/OU=Certificate Authority/CN=fortinet-ca2/emailAddress=support@fortinet.com
[__SSL_info_callback:310] SSL Alert write: fatal unknown CA
[__SSL_info_callback:320] error
[__SSL_info_callback:334] Error error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed
[OFTP_try_accept_SSL_connection:1686 192.168.14.20] SSL accept failed
The solution is to download these two CA certificates (CA_Cert_1 & CA_Cert_2) and import them to the FortiAnalyzer (System Setting > Certificates > CA Certificates).
TCP connection issue with FortiAnalyzer
Long time after FortiWeb sends logs to FortiAnalyzer, sometimes we may encounter the issue that FortiAnalyzer cannot receive new logs from FortiWeb.
-
Use diagnose commands on FortiWeb to analyze:
diagnose debug application oftp 7
diagnose debug enable
Logs are not sent out and the queue is full if seeing the following:
[OFTP][WARN](log_oftp.c:1006): queue[IP_ADDRESS] full: fd=14, discard oldest one!
-
Capture packets on FortiWeb corresponding interface (the interface connecting to FortiAnalyzer), and in the packets there might be.
Many [TCP ZeroWindow] (Win=0) tagged to TCP ACK packets sent from FortiAnalyzer to FortiWeb.
It means FortiAnalyzer is informing FortiWeb to stop sending data because full cache (Win=0) on FortiAnalyzer.
Many TCP Dup Ack from FortiAnalyzer and TCP Retransmission from FortiWeb after FortiWeb sent TLS application data to FortiAnalyzer.
It means FortiWeb sent the logs but received no ACK from FortiAnalyzer.
Suggest to reboot FortiAnalyzer to re-establish new connection between FortiWeb and FortiAnalyzer.
Packet log of attacks is enabled on FortiWeb but they are not displayed on FortiAnalyzer
When a feature is enabled in FortiWeb' GUI Log&Report > Log Config > Other Log Settings > Retain Packet Payload For, the attack packet’s payload that buffered and parsed by HTTP parser will be displayed in attack logs and sent to FortiAnalyzer.
It's an unobvious place on FortiAnalyzer to see such packet payload. Please check FortiAnalyzer > Log View > FortiWeb > Application Attack Prevention > log detail of an attack log. Packet headers and raw data are available by clicking the Data icon.
