Fortinet white logo
Fortinet white logo

Cookbook

Troubleshooting methodologies

Troubleshooting methodologies

The sections in this topic provide an overview of how to prepare to troubleshoot problems in FortiGate. They include verifiying your user permissions, establishing a baseline, defining the problem, and creating a plan.

Verify user permissions

Before you begin troubleshooting, verify the following:

  • You have administrator privileges for the FortiGate.
  • The FortiGate is integrated into your network.
  • The operation mode is configured.
  • The system time, DNS settings, administrator password, and network interfaces are configured.
  • Firmware, FortiGuard AntiVirus, FortiGuard Application Control, and FortiGuard IPS are up to date.
Note

If you are using a FortiGate that has virtual domains (VDOMs) enabled, you can often troubleshoot within your own VDOM. However, you should inform the super_admin for the FortiGate that you willl be performing troubleshooting tasks.

You may also need access to other networking equipment, such as switches, routers, and servers, to carry out tests. If you do not have access to this equipment, contact your network administrator for assistance.

Establish a baseline

FortiGate operates at all layers of the OSI model. For this reason, troubleshooting can be complex. Establishing baseline parameters for your system before a problem occurs helps to reduce the complexity when you need to troubleshoot.

A best practice is to establish and record the normal operating status. Regular operation data shows trends, and allows you to see where changes occur when problems arise. You can gather this data by using logs and SNMP tools to monitor the system performance or by regularly running information gathering commands and saving the output.

note icon

You should back up your FortiOS configuration on a regular basis even when you are not troubleshooting. You can restore the backed up configuration as needed to save time recreating it from the factory default settings.

Use the following CLI commands to obtain normal operating data for a FortiGate:

get system status

Displays firmware versions and FortiGuard engine versions, and other system information.

get system performance status

Displays CPU and memory states, average network usage, average sessions and session setup rate, viruses caught, IPS attacks blocked, and uptime.

get hardware memory

Displays information about memory.

get system session status

Displays total number of sessions.

get router info routing-table all

Displays all the routes in the routing table, including their type, source, and other useful data.

get ips session

Displays memory used and maximum amount available to IPS as well as counts

get webfilter ftgd-statistics

Displays a list of FortiGuard related counts of status, errors, and other data.

diagnose sys session list

Displays the list of current detailed sessions.

show sys dns

Displays the configured DNS servers.

diagnose sys ntp status

Displays information about NTP servers.

You can run any commands that apply to your system for information gathering. For example, if you have active VPN connections, use the get vpn series of commands to get more information about them.

Use execute tac report to get an extensive snapshot of your system. This command runs many diagnostic commands for specific configurations. It also records the current state of each feature regardless of the features deployed on your FortiGate. If you need to troubleshoot later, you can run the same command again and compare the differences to identify any suspicious output.

Define the problem

The following questions are intended to compare the current behavior of the FortiGate with normal operations to help you define the problem. Be specific with your answers. After you define the problem, search for a solution in the troubleshooting scenarios section, and then create a plan to resolve it.

What is the problem?

The problem being observed may not be the actual problem. You should determine where the problem lies before starting to troubleshoot the FortiGate.

Was the device working before?

If the device never worked, it might be defective. For more information, see Troubleshooting your installation.

Can the problem be reproduced?

If the problem is intermittent, it may be dependent on system load.

Intermittent problems are challenging to troubleshoot because they are difficult to reproduce.

What has changed?

Use the FortiGate event log to identify possible configuration changes.

There may be changes in the operating environment. For example, there might be a gradual increase in load as more sites are forwarded through the firewall.

If something has changed, roll back the change and assess the impact.

What is the scope of the problem?

After you isolate the problem, determine what applications, users, devices, and operating systems the problem affects.

The following questions are intended to narrow the scope of the problem and identify what to check during troubleshooting. The more factors you can eliminate, the less you need to check. For this reason, be as specific and accurate as possible when gathering information.

  • What is not working?
  • Is more than one thing not working?
  • Is it partly working? If so, what parts are working?
  • Is it a connectivity issue for the entire device, or is there an application that isn’t reaching the Internet?
  • Where did the problem occur?
  • When did the problem occur and to which users or groups of users?
  • What components are involved?
  • What applications are affected?
  • Can you use a packet sniffer to trace the problem?
  • Can you use system debugging or look in the session table to trace the problem?
  • Do any of the log files indicate a failure has occurred?

Create a troubleshooting plan

After you define the problem and its scope, develop a troubleshooting plan.

Create checklist

Make a list all the possible causes of the problem and how you can test for each cause.

Create a checklist to keep track of what has been tried and what is left to test. Checklists are useful when more than one person is performing troubleshooting tasks.

Obtain the required equipment

Testing your solution may require additional networking equipment, computers, or other devices.

Network administrators usually have additional networking equipment available to loan you, or a lab where you can bring the FortiGate unit to test.

If you do not have access to equipment, check for shareware applications that can perform the same tasks. Often, there are software solutions you can use when hardware is too expensive.

Consult Fortinet troubleshooting resources

After the checklist is created, refer to the troubleshooting scenarios sections to assist with implementing your plan. See Troubleshooting scenarios.

Gather information for technical support

If you still require technical assistance after the plan is implemented, be prepared to provide Fortinet technical support with following information:

  • Firmware build version (use the get system status command)
  • Network topology diagram
  • Recent configuration file
  • Recent debug log (optional)
  • Summary of troubleshooting steps you have taken and the results.
Note

Do not provide the output from the exec tac report unless the support team requests it. The output from this command is very large and is not required in many cases.

Contact technical support

Before contacting technical support, ensure you have login access (preferably with full read/write privileges) to all networking devices that could be relevant to troubleshooting.

If you are using VMs, be prepared to have someone who can log in to the virtual hosting platform in case it is necessary to check and possibly modify resource allocation.

For information about contacting technical support, go to FortiCare Support Service page.

Troubleshooting methodologies

Troubleshooting methodologies

The sections in this topic provide an overview of how to prepare to troubleshoot problems in FortiGate. They include verifiying your user permissions, establishing a baseline, defining the problem, and creating a plan.

Verify user permissions

Before you begin troubleshooting, verify the following:

  • You have administrator privileges for the FortiGate.
  • The FortiGate is integrated into your network.
  • The operation mode is configured.
  • The system time, DNS settings, administrator password, and network interfaces are configured.
  • Firmware, FortiGuard AntiVirus, FortiGuard Application Control, and FortiGuard IPS are up to date.
Note

If you are using a FortiGate that has virtual domains (VDOMs) enabled, you can often troubleshoot within your own VDOM. However, you should inform the super_admin for the FortiGate that you willl be performing troubleshooting tasks.

You may also need access to other networking equipment, such as switches, routers, and servers, to carry out tests. If you do not have access to this equipment, contact your network administrator for assistance.

Establish a baseline

FortiGate operates at all layers of the OSI model. For this reason, troubleshooting can be complex. Establishing baseline parameters for your system before a problem occurs helps to reduce the complexity when you need to troubleshoot.

A best practice is to establish and record the normal operating status. Regular operation data shows trends, and allows you to see where changes occur when problems arise. You can gather this data by using logs and SNMP tools to monitor the system performance or by regularly running information gathering commands and saving the output.

note icon

You should back up your FortiOS configuration on a regular basis even when you are not troubleshooting. You can restore the backed up configuration as needed to save time recreating it from the factory default settings.

Use the following CLI commands to obtain normal operating data for a FortiGate:

get system status

Displays firmware versions and FortiGuard engine versions, and other system information.

get system performance status

Displays CPU and memory states, average network usage, average sessions and session setup rate, viruses caught, IPS attacks blocked, and uptime.

get hardware memory

Displays information about memory.

get system session status

Displays total number of sessions.

get router info routing-table all

Displays all the routes in the routing table, including their type, source, and other useful data.

get ips session

Displays memory used and maximum amount available to IPS as well as counts

get webfilter ftgd-statistics

Displays a list of FortiGuard related counts of status, errors, and other data.

diagnose sys session list

Displays the list of current detailed sessions.

show sys dns

Displays the configured DNS servers.

diagnose sys ntp status

Displays information about NTP servers.

You can run any commands that apply to your system for information gathering. For example, if you have active VPN connections, use the get vpn series of commands to get more information about them.

Use execute tac report to get an extensive snapshot of your system. This command runs many diagnostic commands for specific configurations. It also records the current state of each feature regardless of the features deployed on your FortiGate. If you need to troubleshoot later, you can run the same command again and compare the differences to identify any suspicious output.

Define the problem

The following questions are intended to compare the current behavior of the FortiGate with normal operations to help you define the problem. Be specific with your answers. After you define the problem, search for a solution in the troubleshooting scenarios section, and then create a plan to resolve it.

What is the problem?

The problem being observed may not be the actual problem. You should determine where the problem lies before starting to troubleshoot the FortiGate.

Was the device working before?

If the device never worked, it might be defective. For more information, see Troubleshooting your installation.

Can the problem be reproduced?

If the problem is intermittent, it may be dependent on system load.

Intermittent problems are challenging to troubleshoot because they are difficult to reproduce.

What has changed?

Use the FortiGate event log to identify possible configuration changes.

There may be changes in the operating environment. For example, there might be a gradual increase in load as more sites are forwarded through the firewall.

If something has changed, roll back the change and assess the impact.

What is the scope of the problem?

After you isolate the problem, determine what applications, users, devices, and operating systems the problem affects.

The following questions are intended to narrow the scope of the problem and identify what to check during troubleshooting. The more factors you can eliminate, the less you need to check. For this reason, be as specific and accurate as possible when gathering information.

  • What is not working?
  • Is more than one thing not working?
  • Is it partly working? If so, what parts are working?
  • Is it a connectivity issue for the entire device, or is there an application that isn’t reaching the Internet?
  • Where did the problem occur?
  • When did the problem occur and to which users or groups of users?
  • What components are involved?
  • What applications are affected?
  • Can you use a packet sniffer to trace the problem?
  • Can you use system debugging or look in the session table to trace the problem?
  • Do any of the log files indicate a failure has occurred?

Create a troubleshooting plan

After you define the problem and its scope, develop a troubleshooting plan.

Create checklist

Make a list all the possible causes of the problem and how you can test for each cause.

Create a checklist to keep track of what has been tried and what is left to test. Checklists are useful when more than one person is performing troubleshooting tasks.

Obtain the required equipment

Testing your solution may require additional networking equipment, computers, or other devices.

Network administrators usually have additional networking equipment available to loan you, or a lab where you can bring the FortiGate unit to test.

If you do not have access to equipment, check for shareware applications that can perform the same tasks. Often, there are software solutions you can use when hardware is too expensive.

Consult Fortinet troubleshooting resources

After the checklist is created, refer to the troubleshooting scenarios sections to assist with implementing your plan. See Troubleshooting scenarios.

Gather information for technical support

If you still require technical assistance after the plan is implemented, be prepared to provide Fortinet technical support with following information:

  • Firmware build version (use the get system status command)
  • Network topology diagram
  • Recent configuration file
  • Recent debug log (optional)
  • Summary of troubleshooting steps you have taken and the results.
Note

Do not provide the output from the exec tac report unless the support team requests it. The output from this command is very large and is not required in many cases.

Contact technical support

Before contacting technical support, ensure you have login access (preferably with full read/write privileges) to all networking devices that could be relevant to troubleshooting.

If you are using VMs, be prepared to have someone who can log in to the virtual hosting platform in case it is necessary to check and possibly modify resource allocation.

For information about contacting technical support, go to FortiCare Support Service page.