Troubleshooting methodologies

This section explains how to prepare for troubleshooting, create a troubleshooting plan, and where to find additional resources.

The following topics are covered:

• Ensure you have administrator-level access to required equipment
• Establish a baseline
• Define the problem
• Create a troubleshooting plan
• Obtain any required equipment
• Consult Fortinet resources

Ensure you have administrator-level access to required equipment

Before troubleshooting your FortiGate, you will need administrator access to the equipment. If you're a client on a FortiGate that has virtual domains (VDOMs) enabled, you can often troubleshoot within your own VDOM. However, you should inform the super admin for the FortiGate that you'll be performing troubleshooting tasks.

Also, you may need access to other networking equipment, such as switches, routers, and servers, to carry out tests. If you don't have access to this equipment, contact your network administrator for assistance.

Establish a baseline

A FortiGate operates at all layers of the OSI model. For this reason, troubleshooting problems can become complex. Establishing baseline parameters for your system before a problem occurs helps to reduce the complexity when you need to troubleshoot.

Many of the guiding questions in the following sections serve to compare the current problem situation to normal operation on your FortiGate. A best practice is to establish and record the normal operating status. Regular operation data shows trends, and allows you to see when changes occur and when there may be a problem. You can gather this data by using logs and SNMP tools to monitor the system performance or by regularly running information gathering commands and saving the output.

Back up your FortiOS configuration on a regular basis. This is a good practice and not only for troubleshooting. You can restore the backed up configuration as needed and save the time and effort of recreating it from the factory default settings.

You can use the following CLI commands to obtain normal operating data for a FortiGate:

get system status	Displays firmware versions and FortiGuard engine versions, and other system information
get system performance status	Displays CPU and memory states, average network usage, average sessions and session setup rate, virus caught, IPS attacks blocked, and uptime
get hardware memory	Displays information about memory
get system session status	Displays total number of sessions
get router info routing-table all	Displays all the routes in the routing table, including their type, source, and other useful data
get ips session	Displays memory used and maximum amount available to IPS as well and counts
get webfilter ftgd-statistics	Displays a list of FortiGuard related counts of status, errors, and other data
diagnose sys session list	Displays the list of current detailed sessions
show sys dns	Displays the configured DNS servers
diagnose sys ntp status	Displays information about NTP servers

These commands are just a sample. You can run any commands for information gathering that apply to your system. For example, if you have active VPN connections, use the get vpn series of commands to get more information about them.

To see an extensive snapshot of your system, you can use the execute tac report command. This command runs many diagnostic commands for specific configurations. Regardless of the features deployed on your FortiGate, this command records the current state of each feature. Then, if you need to perform troubleshooting later, you can run the same command again and compare the differences to quickly identify any suspicious output.

Define the problem

The following questions help you define the problem. Be as specific as possible with your answers. Once you define the problem, you can search for a solution and then create a plan for how to solve it.

• What is the problem?

  The problem being observed is not necessarily the actual problem. You should determine where the problem lies before starting to troubleshoot the FortiGate.

• Was the device working before?

  If the device never worked, it might be defective. For more information, see Troubleshooting your FortiGate Installation in the Getting Started chapter.

• Can you reproduce the problem ?

  If the problem is intermittent, it may be dependent on system load. Note that it may be difficult to troubleshoot an intermittent problem because it's difficult to reproduce.

• What has changed?

  Don't assume that nothing has changed in the network. Use the FortiGate event log to identify any possible configuration changes. There may be changes in the operating environment. For example, there might be a gradual increase in load as more sites are forwarded through the firewall. If something has changed, roll back the change and assess the impact.

• What is the scope of the problem?

  After you isolate the problem, determine what applications, users, devices, and operating systems the problem affects.

  • What's not working? Be specific.
  • Is there more than one thing that isn't working?
  • Is it partly working? If so, what parts are working?
  • Is it a connectivity issue for the entire device, or is there an application that isn’t reaching the Internet?
  • Where did the problem occur?
  • When did the problem occur and to which users or groups of users?
  • What components are involved?
  • What applications are affected?
  • Can you use a packet sniffer to trace the problem?
  • Can you use system debugging or look in the session table to trace the problem?
  • Do any of the log files indicate a failure has occurred?

The answers to these questions help you narrow down the problem and identify what you should check during your troubleshooting. The more things you can eliminate, the fewer things you need to check during troubleshooting. For this reason, be as specific and accurate as you can when you gather information.

Create a troubleshooting plan

Once you define the problem and gather facts, you can create a troubleshooting plan to solve the problem.

You should list all possible causes of the problem and how you can test for each cause.

The plan acts as a checklist so that you know what you've tried and what's left to check. This is also important to have if more than one person is performing troubleshooting tasks.

Be ready to add to your plan, as needed.

Providing supporting elements

If you contact Fortinet's Technology Assistance Center (TAC), be prepared to provide the following information:

• Firmware build version (use the get system status command)
• Network topology diagram
• Recent configuration file
• Recent debug log (optional)
• Summary of troubleshooting steps that you've already taken and the results.

Don't provide the output from the exec tac report unless TAC requests it. The output from this command is very large and isn't required in many cases.

Obtain any required equipment

To test your solution, you may require additional networking equipment, computers, or other equipment.

Network administrators usually have additional networking equipment available to loan you, or a lab where you can bring the FortiGate unit to test.

If you don't have access to equipment, check for shareware applications that can perform the same tasks. Often, there are software solutions that you can use when hardware is too expensive.

Consult Fortinet resources

After you define your problem, create a plan to find a solution, and carry out that plan. If you can't resolve the problem, see Troubleshooting resources.