FortiNDR troubleshooting tips

For more information about the CLI commands below, please see the FortiNDR CLI Reference Guide.

Best practices:

Issue Type	Recommendations / Possible cause	CLI command	Comments
General hang issue and disconnect issues	Reload all services to see if the issue is still reproducible	exec reload
CPU usage consistently above 85%	Turn off feature learning to save more resources	exec learner off
GUI not responsive/DB related error (and you can afford to lose all current data)	If you installed an interim build (other than GA) and are willing to wipe all db records	exec db restore	Run exec reload to see if issue is still reproducible
GUI not responsive/DB related error (and you to make a best effort to keep your data)	If you installed an interim build (other than GA) and cannot wipe all db records	diagnose system db	Patches db at best efforts.
General Issues	Retrieve and record all information	get sys status	If you are seeing high CPU and MEM usage, please consider provisioning more resources.
General VM issues	Retrieve and record all information for VMs	diag sys vm	Observe for any FDS code other than 200, and if not 200, please check connections to FDN and license status.

Recommended Debug Setup:

• A syslog server for FortiNDR events log as the GUI only has 1 days events.
• A TFTP server for PCAP capture transfer.

General Debug Logs Retrieval

Scenario	CLI
Collect all crash logs from the first day FortiNDR started	diagnose debug crashlog <crash_log_date>
Record kernel related logs from the bootup and save it to a file	diagnose debug kernel display

File scanning related issues

The following troubleshooting tips are intended to diagnose the error message: File Not Accepted (Client side shows files are submitted but NDR does not have details of file).

To perform a general check:

1. Check and record network conditions from the FortiNDR server to file submitting clients using the following CLI commands:
   • exec ping
   • exec traceroute
2. Make sure all KDBs are updated. For example, no pending updates, no out of date db and no updating.
3. Try submitting a lower throughput, (no archive file type, smaller file size) to see if it is still reproducible.
4. Follow the PCAP dumping guide to dump files from port1 or port2 to make sure the traffic is there. Open dapture pcap with Wireshark to see if there are any redline/blacklines from Wireshark default filter setting which indicates bad network traffic quality. From previous troubleshooting experience, this is the most frequent cause of File Not Accepted.

Troubleshooting HTTP2 issues from FortiGate v7.0 onwards:

Recommendation	Run the following CLI command:
Record output and check for errors	diagnose system csf global
Record output and make sure status is authorized	diagnose system csf upstream
Collect logs	diag debug enable and diag debug application csfd 7

Manual Upload/API Submission/FortiSandbox Integration

For all issues:

Start with a single file upload and fetch results from the same subnet as directed from where the client resides. See Appendix A: API guide.

To verify the process is successful:

If a single file submit/fetch is working from the previous step. Run the following CLI commands:

• diag debug enable

and

• diagnose debug application 7

Record all output and look for any non 200 http code or stack traces.

File Submitted but not processed

Collect all the information from the process and record it using the following CLI commands:

• diag debug enable

and

• diagnose debug process <process_name>

Information for support tickets

For an optimal support experience, we recommend including the following information in your support ticket:

Model Name	What Model is your platform
Firmware Version	Which Firmware is your platform
Always Reproducible	True/False
Reproducible Steps	If this issue is reproducible, please include the steps to reproduce this issue.
Actual Result vs Expected Results	What are the expected results and actual results.
Troubleshootingrecommendations used	Please describe the troubleshooting recommendations you attempted and the outcome.
Crash Log output	Run the following CLI command crash_log_date and provide the output. For more information see, diagnose debug.
Kernel Log output	Record the output from diag debug kernel display. For more information see, diagnose debug.
Application Log Output	Specified log output from affected application. For more information see, diagnose debug.
Database Error Log	Record the log output from diag deb database error-log.
FortiNDR Event Log	If configured with FAZ/FortiSIEM, please provide logs from FAZ/FortiSIEM. Otherwise, please include a screen shot from Event tab.
Client-Side Log	When FortiNDR acts as a server role (ICAP, OFTP and HTTP2 inline blocking etc.), please also include logs from the client side.
System Status	Provide the CPU and MEM usage when the issue occurs.
FortiNDR configuration	How many ports are configured/used? How many applications are used, and how are they configured? If you have a backup configuration file, please include it in your support ticket. For more information, see Backup or restore the system configuration.
FortiNDR Load	For configured applications, how much load are applied. For example: For AV, how many files are being sent to FortiNDR, and what is the file size distributions and file type distributions? For NDR, what is the bandwidth for traffic and what is the distribution of application type?
For non-hardware platform:
Virtual machine provisioning	How many CPU/Mems are provisioned? How was the disk provisioned: Thin, Thick etc.
Hosting Hardware specifications	CPU Model Number, RAM Channels, DISK model number.
Physical hosting hardware load	A historical log of host hardware status to indicate CPU and MEM usage