Fortinet black logo

Handbook

FortiOS diagnostics

6.0.0
Copy Link
Copy Doc ID 4afb0436-a998-11e9-81a4-00505692583a:178055
Download PDF

FortiOS diagnostics

FortiOS has a collection of diagnostic commands that you can use to troubleshoot and monitor the performance of your network. The get and diagnose CLI commands are the two main groups of diagnostic commands. Both commands display information about system resources, connections, and settings that allow you to locate problems or monitor system performance.

This topic includes diagnostics commands to help with:

Date and time

The system date and time are important for FortiGuard services, logging events, and sending alerts. The wrong time makes the log entries confusing and difficult to use.

Use Network Time Protocol (NTP) to set the date and time, if possible. This is an automatic method that doesn't require manual intervention. However, you must ensure that the port is allowed through the firewalls on your network. FortiToken synchronization requires NTP in many situations.

How to set the date and time - GUI
  1. Go to the System Information widget on the Dashboard. The date and time are displayed next to System Time.
  2. To adjust the date and time settings, go to System > Settings.
  3. In System Time, you can set the time zone, date and time, and select NTP usage.
How to check the date and time - CLI

You can check the date and time using the CLI commands execute date and execute time.

config system global

set timezone <integer>

end

config system ntp

set type custom

config ntpserver

edit 1

set server “ntp1.fortinet.net”

next

edit 2

set server “ntp2.fortinet.net”

next

end

set ntpsync enable

set syncinterval 60

end

Use the set timezone ? command to display a list of timezones and the integers that represent them.

Resource usage

Each program that runs on a computer has one or more processes associated with it. For example, if you open a Telnet program, it has an associated Telnet process. The same is true in FortiOS. All processes share the system resources in FortiOS, including memory and CPU.

Use the get system performance status command to show the FortiOS performance status.

Sample output:

FGT# get system performance status

CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

Memory: 4050332k total, 527148k used (13%), 3381312k free (83%), 141872k freeable (3%)

Average network usage: 41 / 28 kbps in 1 minute, 54 / 44 kbps in 10 minutes, 42 / 34 kbps in 30 minutes

Average sessions: 33 sessions in 1 minute, 48 sessions in 10 minutes, 38 sessions in 30 minutes

Average session setup rate: 0 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes

Virus caught: 0 total in 1 minute

IPS attacks blocked: 0 total in 1 minute

Uptime: 0 days, 22 hours, 59 minutes

Monitor the CPU and memory usage of internal processes, using the following command:

get system performance top <delay> <max_lines>

The data that the command lists includes the name of the daemon, the process ID, whether the process is sleeping or running, the CPU use percentage, and the memory use percentage.

Sample output:

get system performance top 10 100

Run Time: 0 days, 23 hours and 4 minutes

0U, 0N, 0S, 100I, 0WA, 0HI, 0SI, 0ST; 3955T, 3298F

httpsd 212 S 0.4 0.6

forticron 169 S 0.4 0.4

newcli 4054 R 0.4 0.2

reportd 174 S 0.0 1.4

pyfcgid 325 S 0.0 0.8

cmdbsvr 141 S 0.0 0.7

miglogd 160 S 0.0 0.6

httpsd 211 S 0.0 0.6

src-vis 180 S 0.0 0.6

pyfcgid 327 S 0.0 0.6

pyfcgid 328 S 0.0 0.6

pyfcgid 329 S 0.0 0.6

httpsd 162 S 0.0 0.5

cw_acd 189 S 0.0 0.5

httpsd 3998 S 0.0 0.5

httpsd 4050 S 0.0 0.5

updated 176 S 0.0 0.5

httpsd 4052 S 0.0 0.4

miglogd 203 S 0.0 0.4

miglogd 204 S 0.0 0.4

Proxy operation

Monitor proxy operations, using the following command:

diagnose test application <application> <option>

o display a list of available <application> values, enter:

diagnose test application ?

The <option> value depends on the application value that you use in the command. To display a list of available <option> values, enter:

diagnose test application <application> ?

For example, if the application is http, the CLI command that displays the <option> values is:

diagnose test application http ?

Hardware NIC

To monitor hardware network operations, use the following command:

diagnose hardware deviceinfo nic <interface>

The information that this command shows is important because errors at the interface indicate data link or physical layer issues which may impact the performance of the FortiGate.

The following example shows a sample output when you set <interface> to lan:

System_Device_Name lan

Current_HWaddr 00:09:0f:68:35:60

Permanent_HWaddr 00:09:0f:68:35:60

State up

Link up

Speed 100

Duplex full

[……]

Rx_Packets=5685708

Tx_Packets=4107073

Rx_Bytes=617908014

Tx_Bytes=1269751248

Rx_Errors=0

Tx_Errors=0

Rx_Dropped=0

Tx_Dropped=0

[……]

The diagnose hardware deviceinfo nic command displays a list of error names and values that are related to hardware. The following table describes possible hardware errors:

Field

Description

Rx_Errors = rx error count

Bad frame was marked as error by PHY

Rx_CRC_Errors +

Rx_Length_Errors -

Rx_Align_Errors

This error is only valid in 10/100M mode

Rx_Dropped or

Rx_No_Buffer_Count

Running out of buffer space

Rx_Missed_Errors

Equals Rx_FIFO_Errors + CEXTERR (Carrier Extension Error Count); only valid in 1000M mode, which is marked by PHY

Tx_Errors = Tx_Aborted_Errors

ECOL (Excessive Collisions Count); only valid in half-duplex mode

Tx_Window_Errors

Late Collisions (LATECOL) Count

Late collisions are collisions that occur after 64-byte time into the transmission of the packet while working in 10 to 100 Mb/s data rate and 512-byte time into the transmission of the packet while working in the 1,000 Mb/s data rate. This register only increments if transmits are enabled and the device is in half-duplex mode.

Rx_Dropped

See Rx_Errors

Tx_Dropped

Not defined

Collisions

Total number of collisions experienced by the transmitter; valid in half-duplex mode

Rx_Length_Errors

Transmission length error

Rx_Over_Errors

Not defined

Rx_CRC_Errors

Frame CRC error

Rx_Frame_Errors

Same as Rx_Align_Errors

This error is only valid in 10/100M mode.

Rx_FIFO_Errors

Same as Rx_Missed_Errors - a missed packet count

Tx_Aborted_Errors

See Tx_Errors

Tx_Carrier_Errors

The PHY should assert the internal carrier sense signal during every transmission. Failure to do so may indicate that the link has failed or the PHY has an incorrect link configuration. This register only increments if transmits are enabled. This register isn't valid in internal SerDes 1 mode (TBI mode for the 82544GC/EI) and is valid only when the Ethernet controller is operating at full duplex.

Tx_FIFO_Errors

Not defined

Tx_Heartbeat_Errors

Not defined

Tx_Window_Errors

See LATECOL

Tx_Single_Collision_Frames

Counts the number of times that a successfully transmitted packet encountered a single collision

The value increments only if transmits are enabled and the Ethernet controller is in half-duplex mode.

Tx_Multiple_Collision_Frames

A Multiple Collision Count which indicates the number of times that a transmit encountered more than one collision, but less than 16. The value increments only if transmits are enabled and the Ethernet controller is in half-duplex mode.

Tx_Deferred

Counts defer events

A deferred event occurs when the transmitter can't immediately send a packet due to the medium being busy because another device is transmitting, the IPG timer hasn't expired, half-duplex deferral events are occurring, XOFF frames are being received, or the link isn't up. This register only increments if transmits are enabled. This counter doesn't increment for streaming transmits that are deferred due to TX IPG.

Rx_Frame_Too_Longs

The Rx frame is oversized

Rx_Frame_Too_Shorts

The Rx frame is too short

Rx_Align_Errors

This error is only valid in 10/100M mode

Symbol Error Count

Counts the number of symbol errors between reads - SYMERRS.

The count increases for every bad symbol that's received, whether or not a packet is currently being received and whether or not the link is up. This register increments only in internal SerDes mode.

Traffic trace

Traffic tracing allows you to follow a specific packet stream. This is useful to confirm that packets are taking the route you expect them to take on your network.

View the characteristics of a traffic session though specific security policies using:

diagnose sys session

Trace per-packet operations for flow tracing using:

diagnose debug flow

Trace per-Ethernet frame using:

diagnose sniffer packet

Trace a route from a FortiGate to a destination IP address using:

# execute traceroute www.fortinet.com

traceroute to www.fortinet.com (66.171.121.34), 32 hops max, 84 byte packets

1 172.20.120.2 0.637 ms 0.653 ms 0.279 ms

2 209.87.254.221 <static-209-87-254-221.storm.ca> 2.448 ms 2.519 ms 2.458 ms

3 209.87.239.129 <core-2-g0-2.storm.ca> 2.917 ms 2.828 ms 9.324 ms

4 209.87.239.199 <core-3-bdi1739.storm.ca> 13.248 ms 12.401 ms 13.009 ms

5 216.66.41.113 <v502.core1.tor1.he.net> 17.181 ms 12.422 ms 12.268 ms

6 184.105.80.9 <100ge1-2.core1.nyc4.he.net> 21.355 ms 21.518 ms 21.597 ms

7 198.32.118.41 <ny-paix-gni.twgate.net> 83.297 ms 84.416 ms 83.782 ms

8 203.160.228.217 <217-228-160-203.TWGATE-IP.twgate.net> 82.579 ms 82.187 ms 82.066 ms

9 203.160.228.229 <229-228-160-203.TWGATE-IP.twgate.net> 82.055 ms 82.455 ms 81.808 ms

10 203.78.181.2 82.262 ms 81.572 ms 82.015 ms

11 203.78.186.70 83.283 ms 83.243 ms 83.293 ms

12 66.171.127.177 84.030 ms 84.229 ms 83.550 ms

13 66.171.121.34 <www.fortinet.com> 84.023 ms 83.903 ms 84.032 ms

14 66.171.121.34 <www.fortinet.com> 83.874 ms 84.084 ms 83.810 ms

Session table

A session is a communication channel between two devices or applications across the network. Sessions allow FortiOS to inspect and act on a sequential group of packets in a session all together instead of inspecting each packet individually. Each of these sessions has an entry in the session table that includes important information about the session.

Use as a tool

Session tables are useful troubleshooting tools because they allow you to verify open connections. For example, if you have a web browser open to browse the Fortinet website, you would expect a session entry from your computer, on port 80, to the IP address for the Fortinet website. Another troubleshooting method is if there are too many sessions for FortiOS to process, you can examine the session table for evidence why this is happening.

You can view the FortiGate session table from either the FortiGate GUI or CLI. The most useful troubleshooting data comes from the CLI. The session table in the GUI also provides useful summary information, particularly the current policy number that the session is using.

GUI session information

You can view session information by going to the FortiView page. Read more about FortiView consoles in the Handbook's FortiView chapter.

How to find which security policy a specific connection is using

Every program and device on your network must have a communication channel, or session, open to pass information. The FortiGate manages these sessions with features such as traffic shaping, antivirus scanning, and blocking known bad web sites. Each session has an entry in the session table.

You may want to find information for a specific session for troubleshooting. For example, if a secure web browser session isn't working properly, you can check the session table to ensure the session is still active and going to the proper address. The session table can also tell you the security policy number it matches, so you can check what's happening in that policy.

  1. Know your connection information.

    You need to be able to identify the session you want. For this, you need the source IP address (usually your computer), the destination IP address (if you have it), and the port number which is determined by the program that you're using. Some commons ports are:

    • Port 80 (HTTP for web browsing)
    • Port 80 (HTTP for web browsing)
    • Port 80 (HTTP for web browsing)
    • Port 80 (HTTP for web browsing)
  2. Find your session and policy ID.

    Go to FortiView > All Sessions. Find your session by finding your source IP address, destination IP address (if you have it), and port number. The policy ID is listed after the destination information. If the list of sessions is very long, you can filter the list to make it easier to find your session.

  3. When there are many sessions, use a filter to help you find your session.

    If there are multiple pages of sessions, it's difficult to find a single session. You can use a filter to block out sessions that you don’t want. Click the search icon on the column heading to select the filter. Select Source IP and enter your source IP address. Now, only sessions that originate from your IP address are displayed in the session table. If the list is still too long, you can do the same for the Source port. That makes it easy to find your session and the security policy ID.

CLI session information

The session table output that the diagnose sys session list command generates is very large. You can use filters to display only the session data that you're interested in.

An entry is placed in the session table for each traffic session passing through a security policy. The following command lists the information for a session in the table:

diagnose sys session list

The filter option displays specific information, for example:

diagnose sys session filter <option>

The values for <option> include the following:

clear

Clear session filter

dintf

Destination interface

dport

Destination port

dst

Destination IP address

duration

Duration of the session

expire

Expire

negate

Inverse filter

nport

NAT'd source port

nsrc

NAT'd source ip address

policy

Policy ID

proto

Protocol number

proto-state

Protocol state

session-state1

Session state1

session-state2

Session state2

sintf

Source interface

sport

Source port

src

Source IP address

vd

Index of virtual domain, -1 matches all

Even though UDP is a sessionless protocol, the FortiGate keeps track of the following states:

  • When UDP reply doesn't have a value of 0
  • When UDP reply has a value of 1

The following table displays firewall session states from the session table:

State

Description

log

Session is being logged

local

Session is originated from or destined for local stack

ext

Session is created by a firewall session helper

may_dirty

Session is created by a policy

For example, the session for ftp control channel will have this state but ftp data channel won't. This is also seen when NAT is enabled.

ndr

Session will be checked by IPS signature

nds

Session will be checked by IPS anomaly

br

Session is being bridged (TP) mode

Firewall session setup rate

The number of sessions that can be established in a set period of time is useful information. A session is an end-to-end TCP/IP connection for communication with a limited lifespan. If you record the setup rate during normal operation, when you experience problems you can compare the baseline setup rate to the rate that occurs when you're troubleshooting. This can be a useful step to help you define your problem.

A reduced firewall session setup rate can be the result of a number of things, such as a lack of system resources on the FortiGate or reaching the limit of your session count for your VDOM.

To view your session setup rate method 1 - CLI

FGT# get system performance status

CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

Memory: 4050332k total, 530512k used (13%), 3376844k free (83%), 142976k freeable (3%)

Average network usage: 131 / 90 kbps in 1 minute, 26 / 15 kbps in 10 minutes, 49 / 42 kbps in 30 minutes

Average sessions: 80 sessions in 1 minute, 30 sessions in 10 minutes, 42 sessions in 30 minutes

Average session setup rate: 3 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes

Virus caught: 0 total in 1 minute

IPS attacks blocked: 0 total in 1 minute

Uptime: 1 days, 2 hours, 45 minutes

The information you're looking for is the average sessions section, in the above output. This example shows that there were 80 sessions in 1 minute, or an average of 3 sessions per second. The values for 10 minutes and 30 minutes allow you to take a longer average for a more reliable value if your FortiGate is working at maximum capacity. The smallest FortiGate can have 1,000 sessions established per second across the unit.

Remember that session setup rate is a global command. If you have multiple VDOMs configured with many sessions in each one, the session setup rate per VDOM will be slower than if there are no VDOMs configured.

Finding object dependencies

An administrator may not be permitted to delete a configuration object if there are other configuration objects that depend on it. This command identifies other objects which depend on, or make reference to, the configuration object in question. If an error is displayed that an object is in use and can't be deleted, this command can help identify the source of the problem.

Additionally, if you have a virtual interface with objects that depend on it, you need to find and remove those dependencies before you delete the interface.

CLI method

When you run multiple VDOMs, you use this command in the global configuration only and it searches for the named object in both the most recently used global and VDOM configurations:

diagnose sys cmdb refcnt show <path.object.mkey> <tablename>

For example, to verify which objects a security policy with an ID of 1 refers to, enter the following command:

diagnose sys cmdb refcnt show firewall.policy.policyid 1

To check what is referred to by interface port1, enter the following command:

diagnose sys cmdb refcnt show system.interface.name port1

To show all dependencies for an interface, enter the following command:

diagnose sys cmdb refcnt show system.interface.name <interface name>

Sample output:

entry used by table firewall.address:name '10.98.23.23_host’

entry used by table firewall.address:name 'NAS'

entry used by table firewall.address:name 'all'

entry used by table firewall.address:name 'fortinet.com'

entry used by table firewall.vip:name 'TORRENT_10.0.0.70:6883'

entry used by table firewall.policy:policyid '21'

entry used by table firewall.policy:policyid '14'

entry used by table firewall.policy:policyid '19'

In this example, the interface has dependent objects, including four address objects, one VIP, and three security policies.

GUI method

In the GUI, you can easily check and remove the object dependencies for an interface.

To remove interface object dependencies - GUI
  1. Go to Network > Interfaces. The Ref. column displays the number of objects that refer to this interface.
  2. Select the number in the Ref. column for the interface. A window listing the dependencies appears.
  3. Use these detailed entries to locate and remove object references to this interface. The trash can icon changes from gray when you've removed all object dependencies.
  4. Remove the interface by selecting the check box for the interface, and select Delete.

Flow trace

To trace the flow of packets through the FortiGate, use the following command:

diagnose debug flow trace start

Follow packet flow by setting a flow filter, using this command:

diagnose debug flow {filter | filter6} <option>

If your network uses IPv4, enter filter. If your network uses IPv6, enter filter6.

One of the following variables replaces <option>:

Variable

Description

addr

IPv4 or IPv6 address

clear

clear filter

daddr

destination IPv4 or IPv6 address

dport

destination port

negate

inverse IPv4 or IPv6 filter

port

port

proto

protocol number

saddr

source address

sport

source port

vd

index of virtual domain; -1 matches all

caution icon

diagnose debug flow output is recorded as event log messages and which are then sent to a FortiCloud or a FortiAnalyzer, if connected. Don't run this command longer than necessary, since it generates significant amounts of data.

To start flow monitoring with a specific number of packets - CLI:

diagnose debug flow trace start <N>

To stop flow tracing at any time using - CLI:

diagnose debug flow trace stop

The following example shows the flow trace for a device with an IP address of 203.160.224.97:

diagnose debug enable

diagnose debug flow filter addr 203.160.224.97

diagnose debug flow show function-name enable

diagnose debug flow trace start 100

Flow trace output example - HTTP

To observe the debug flow trace, connect to the web site at the following address:

https://www.fortinet.com

Comment: SYN packet received:

id=20085 trace_id=209 func=resolve_ip_tuple_fast

line=2700 msg="vd-root received a packet(proto=6,

192.168.3.221:1487->203.160.224.97:80) from port5."

SYN sent and a new session is allocated:

id=20085 trace_id=209 func=resolve_ip_tuple line=2799

msg="allocate a new session-00000e90"

Lookup for next-hop gateway address:

id=20085 trace_id=209 func=vf_ip4_route_input line=1543

msg="find a route: gw-192.168.11.254 via port6"

Source NAT, lookup next available port:

id=20085 trace_id=209 func=get_new_addr line=1219

msg="find SNAT: IP-192.168.11.59, port-31925"

direction“

Matched security policy. Check to see which policy this session matches:

id=20085 trace_id=209 func=fw_forward_handler line=317

msg="Allowed by Policy-3: SNAT"

Apply source NAT:

id=20085 trace_id=209 func=__ip_session_run_tuple

line=1502 msg="SNAT 192.168.3.221->192.168.11.59:31925"

SYN ACK received:

id=20085 trace_id=210 func=resolve_ip_tuple_fast line=2700

msg="vd-root received a packet(proto=6, 203.160.224.97:80-

>192.168.11.59:31925) from port6."

Found existing session ID. Identified as the reply direction:

id=20085 trace_id=210 func=resolve_ip_tuple_fast line=2727

msg="Find an existing session, id-00000e90, reply direction"

Apply destination NAT to inverse source NAT action:

id=20085 trace_id=210 func=__ip_session_run_tuple

line=1516 msg="DNAT 192.168.11.59:31925-

>192.168.3.221:1487"

Lookup for next-hop gateway address for reply traffic:

id=20085 trace_id=210 func=vf_ip4_route_input line=1543

msg="find a route: gw-192.168.3.221 via port5"

ACK received:

id=20085 trace_id=211 func=resolve_ip_tuple_fast line=2700

msg="vd-root received a packet(proto=6,

192.168.3.221:1487->203.160.224.97:80) from port5."

Match existing session in the original direction:

id=20085 trace_id=211 func=resolve_ip_tuple_fast line=2727

msg="Find an existing session, id-00000e90, original

direction"

Apply source NAT:

id=20085 trace_id=211 func=__ip_session_run_tuple

line=1502 msg="SNAT 192.168.3.221->192.168.11.59:31925"

Receive data from client:

id=20085 trace_id=212 func=resolve_ip_tuple_fast

line=2700 msg="vd-root received a packet(proto=6,

192.168.3.221:1487->203.160.224.97:80) from port5."

Match existing session in the original direction:

id=20085 trace_id=212 func=resolve_ip_tuple_fast

line=2727 msg="Find an existing session, id-00000e90,

original direction"

Apply source NAT:

id=20085 trace_id=212 func=__ip_session_run_tuple

line=1502 msg="SNAT 192.168.3.221->192.168.11.59:31925"

Receive data from server:

id=20085 trace_id=213 func=resolve_ip_tuple_fast

line=2700 msg="vd-root received a packet(proto=6,

203.160.224.97:80->192.168.11.59:31925) from port6."

Match existing session in reply direction:

id=20085 trace_id=213 func=resolve_ip_tuple_fast

line=2727 msg="Find an existing session, id-00000e90,

reply direction"

Apply destination NAT to inverse source NAT action:

id=20085 trace_id=213 func=__ip_session_run_tuple

line=1516 msg="DNAT 192.168.11.59:31925-

>192.168.3.221:1487"

Flow trace output example - IPsec (policy-based)

id=20085 trace_id=1 msg="vd-root received a packet(proto=1, 10.72.55.240:1->10.71.55.10:8) from internal."

id=20085 trace_id=1 msg="allocate a new session-00001cd3"

id=20085 trace_id=1 msg="find a route: gw-66.236.56.230 via wan1"

id=20085 trace_id=1 msg="Allowed by Policy-2: encrypt"

id=20085 trace_id=1 msg="enter IPsec tunnel-RemotePhase1"

id=20085 trace_id=1 msg="encrypted, and send to 15.215.225.22 with source 66.236.56.226"

id=20085 trace_id=1 msg="send to 66.236.56.230 via intf-wan1“

id=20085 trace_id=2 msg="vd-root received a packet (proto=1, 10.72.55.240:1-1071.55.10:8) from internal."

id=20085 trace_id=2 msg="Find an existing session, id-00001cd3, original direction"

id=20085 trace_id=2 msg="enter IPsec ="encrypted, and send to 15.215.225.22 with source 66.236.56.226“ tunnel-RemotePhase1"

id=20085 trace_id=2 msgid=20085 trace_id=2 msg="send to 66.236.56.230 via intf-wan1"

Packet sniffing and packet capture

When you troubleshoot networks, it helps to look inside the header of the packets. This helps to determine if the packets, route, and destination are all what you expect. Packet capture can also be called a network tap, packet sniffing, or logic analyzing. FortiOS devices can sniff packets using CLI commands or capture packets using the GUI.

Packet sniffing using CLI commands is well-suited for spot checking traffic, but if you have complex filters to enter it can be a lot of work to enter them each time. You can also save the sniffing output. However, you must log to a file and then analyze the file later.

Packet capture in the GUI makes it easy for you to set up multiple filters at once and run only one or two as you need to. You can also use controls to start and stop capturing when you want to. You download packet capture output to your local computer as a *.pcap file. You must use a third party application, such as Wireshark, to read *.pcap files. This method is useful to send information to Fortinet support to help resolve an issue.

The following table presents a comparison between the two methods:

Features

Packet sniffing

Packet capture

Command location

CLI

GUI

Third party software required

puTTY to log plaintext output

Wireshark, or similar application, to read *.pcap files

Read output in plain text file

yes

no

Read output as *.pcap file using Wireshark, or similar application

no

yes

Easily configure single quick and simple filter

yes

no

Record packet interface

yes

no

Configure complex sniffer filters on multiple interface

no

yes

sniff IPv6

hard

easy

sniff non-IP packets

no

yes

Filter packets by protocol and/or port

easy

easy

Filter packets by source and/or destination address

easy

easy

Packet sniffing

If you're running a constant traffic application, such as ping, packet sniffing can tell you if the traffic is reaching the destination, what the port of entry is on the FortiGate unit, if the ARP resolution is correct, and if the traffic is being sent back to the source as expected.

Sniffing packets can also tell you if the FortiGate is silently dropping packets for reasons such as Reverse Path Forwarding (RPF). RPF, also called anti-spoofing, prevents an IP packet from being forwarded if its source IP doesn't belong to a locally attached subnet (local interface) or isn't part of the routing between the FortiGate and another source (static route, RIP, OSPF, BGP). Note that you can disable RPF by turning on asymmetric routing in the CLI (config system settings, set asymroute enable), but this disables stateful inspection on the FortiGate and causes many features to be turned off.

caution icon

If you configure virtual IP addresses on your FortiGate, it will use those addresses instead of the physical IP addresses. You'll notice this when you're sniffing packets because all the traffic uses the virtual IP addresses. This is due to the ARP update that's sent out when the VIP address is configured.

Before you start sniffing packets on the CLI, you should prepare to capture the output to a file. A large amount of data may scroll by and you won't be able to see it without first saving it to a file. One method is to use a terminal program like puTTY to connect to the FortiGate CLI. Once the packet sniffing count is reached, you can end the session and analyze the output in the file.

The general form of the internal FortiOS packet sniffer command is:

diagnose sniffer packet <interface_name> <‘filter’> <verbose> <count> <tsformat>

To stop the sniffer, type CTRL+C.

<interface_name>

The name of the interface to sniff, such as “port1” or “internal”. This can also be “any” to sniff all interfaces.

<‘filter’>

What to look for in the information the sniffer reads. “none” indicates no filtering, and all packets are displayed as the other arguments indicate.

The filter must be inside single quotes (‘).

<verbose>

The level of verbosity as one of:

1 - print header of packets
2 - print header and data from IP of packets
3 - print header and data from Ethernet of packets
4 - print header of packets with interface name

<count>

The number of packets the sniffer reads before stopping. If you don't put a number here, the sniffer will run until you stop it with <CTRL+C>.

<tsformat>

The format of timestamp.
  • a: absolute UTC time, yyyy-mm-dd hh:mm:ss.ms
  • l: absolute LOCAL time, yyyy-mm-dd hh:mm:ss.ms
  • otherwise: relative to the start of sniffing, ss.ms

For a simple sniffing example, enter the CLI command diagnose sniffer packet port1 none 1 3. This displays the next three packets on the port1 interface using no filtering, and verbose level 1. At this verbosity level, you can see the source IP and port, the destination IP and port, action (such as ack), and sequence numbers.

In the output below, port 443 indicates these are HTTPS packets and that 172.20.120.17 is both sending and receiving traffic.

Head_Office_620b # diagnose sniffer packet port1 none 1 3

interfaces=[port1]

filters=[none]

0.545306 172.20.120.17.52989 -> 172.20.120.141.443: psh 3177924955 ack 1854307757

0.545963 172.20.120.141.443 -> 172.20.120.17.52989: psh 1854307757 ack 3177925808

0.562409 172.20.120.17.52988 -> 172.20.120.141.443: psh 4225311614 ack 3314279933

For a more advanced example of packet sniffing, the following commands will report packets on any interface that are traveling between a computer with the host name of “PC1” and a computer with the host name of “PC2”. With verbosity 4 and above, the sniffer trace displays the interface names where traffic enters or leaves the FortiGate unit. Remember to stop the sniffer, type CTRL+C.

FGT# diagnose sniffer packet any "host <PC1> or host <PC2>" 4

or

FGT# diagnose sniffer packet any "(host <PC1> or host <PC2>) and icmp" 4

The following CLI command for a sniffer includes the ARP protocol in the filter which may be useful to troubleshoot a failure in the ARP resolution (for example, PC2 may be down and not responding to the FortiGate ARP requests).

FGT# diagnose sniffer packet any "host <PC1> or host <PC2> or arp" 4

Packet capture

Packet capture tells you what is happening on the network at a low level. This can be very useful for troubleshooting problems, such as:

  • Finding missing traffic
  • Seeing if sessions are setting up properly
  • Locating ARP problems such as broadcast storm sources and causes
  • Confirming which address a computer is using on the network, if they have multiple addresses or are on multiple networks
  • Confirming routing is working as you expect
  • Connecting wireless clients
  • Missing PING packets
  • A particular type of packet is having problems, such as UDP, which is commonly used for streaming video

If you're running a constant traffic application such as ping, packet capture can tell you if the traffic is reaching the destination, how the port enters and exits the FortiGate, if the ARP resolution is correct, and if the traffic is returning to the source as expected. You can also use packet switching to verify that NAT or other configuration is translating addresses or routing traffic the way that you want it to.

Before you start capturing packets, you need to have a good idea of what you're looking for. Capture is used to confirm or deny your ideas about what's happening on the network. If you try capture without a plan to narrow your search, you can end up with too much data to effectively analyze. On the other hand, you need to capture enough packets to really understand all of the patterns and behavior that you're examining.

To use packet capture, the FortiGate must have a disk. You can enable the capture-packet in the firewall policy, using the following CLI commands:

config firewall policy

edit <id>

set capture-packet enable

end

To configure packet capture filters, go to Network > Packet Capture.

When you add a packet capture filter, enter the following information and select OK.

Interface

Select the interface to sniff from the drop-down menu.

You must select one interface. You can't change the interface without deleting the filter and creating a new one, unlike the other fields.

Max Packets to Save

Enter the number of packets to capture before the filter stops.

This number can't be zero. You can halt the capturing before this number is reached.

Enable Filters

Select this option to specify filter fields

Host(s)

Enter the IP address of one or more hosts.

Separate multiple hosts with commas. To enter a range, use a dash without spaces, for example 172.16.1.5-172.16.1.15, or enter a subnet.

Port(s)

Enter one or more ports to capture on the selected interface.

Separate multiple ports with commas. To enter a range, use a dash without spaces, for example 88-90

VLAN(s)

Enter one or more VLANs (if any). Separate multiple VLANs with commas.

Protocol

Enter one or more protocols. Separate multiple protocols with commas. To enter a range, use a dash without spaces, for example 1-6, 17, 21-25.

Include IPv6 Packets

Select this option if you're troubleshooting IPv6 networking, or if your network uses IPv6. Otherwise, leave it disabled.

Include Non-IP Packets

The protocols in the list are all IP based except for ICMP (ping). To capture non-IP based packets, select this feature. Examples of non-IP packets include IPsec, IGMP, ARP, and ICMP.

If you select a filter, you have the option to start and stop packet capture in the edit window, or download the captured packets. You can also see the filter status and the number of packets captured.

You can select the filter and start capturing packets. When the filter is running, the number of captured packets increases until it reaches the Max Packet Count or you stop it. When the filter is running, you can't download the output file.

When the packet capture is complete, you can download the *.pcap file. You must use a third party application, such as Wireshark, to read *,pcap files. This tool provides you with extensive analytics and the full contents of the packets that were captured.

To start, stop, or resume packet capture, use the symbols on the screen. These symbols are the same as those used for audio or video playback. Hover over the symbol to reveal explanatory text. Similarly, to download the *.pcap file, use the download symbol on the screen.

NPU-based interfaces

Many Fortinet products contain network processors, such as NP1, NP2, NP4, and NP6, which means that offloading requirements vary depending on the model.

When you use the NPU-based interfaces, you can see only the initial session setup will be seen using the diagnose debug flow command. If the session is correctly programmed into the ASIC (fastpath), the diagnose debug flow command won't detect the packets that arrive at the CPU. If the NPU functionality is disabled, the CPU detects all the packets. However, you should only this for troubleshooting purposes.

First, obtain the NP4 or NP6 ID and the port numbers, using the following command:

diagnose npu {np4|npu6}list

Sample output:

ID Model Slot Interface

0 On-board port1 fabric1 fabric3 fabric5

1 On-board fabric2 port2 base2 fabric4

Run the following commands:

diagnose npu {np4|npu6}fastpath disable <dev_id>

(where dev_id is the NP4 or NP6 number)

Then, run this command:

diagnose npu {np4|npu6}fastpath-sniffer enable port1

Sample output:

NP4 Fast Path Sniffer on port1 enabled

This causes traffic on port1 of the network processor to be sent to the CPU. this means that you can take a standard sniffer trace and use other diagnose commands, if it's a standard CPU-driven port.

These commands only apply to the newer NP4 and NP6 interfaces.

Debug command

Debug output provides continuous, real-time event information and continues until you stop it or reboot the unit. Debug output can affect system performance and is continually generated even though output might not be displayed in the CLI console.

Debug information that's displayed in the console scrolls in the console display and may prevent you from entering CLI commands, such as, the command to disable the debug display. To turn off debug output as the display scrolls by, press the key to recall the recent diagnose debug command, press backspace, and type “0”, and Enter.

To enable debug output display, use the following command:

diagnose debug enable

Once you enable debug output display, specify the debug information that you require using the following command:

diagnose debug <option> <level>

Debug command options include the following:

enable

Enable debug output

disable

Disable debug output

info

Show active debug level settings

reset

Reset all debug level to default

report

Report for tech support

crashlog

Crash log info

config-error-log

Configure error log info

sql-log-error

SQL log database error info

application

application

kernel

kernel

remote-extender

remote-extender

cli

Debug CLI

cmdb-trace

Trace CLI

rating

Display rating info

authd

Authentication daemon

fsso-polling

FSSO active directory poll module

flow

Trace packet flow in kernel

urlfilter

urlfilter

admin

Admin user

You can set the debug level at the end of the command. For example, typical values are 2 and 3, like in the following commands:

diagnose debug application DHCPS 2

diagnose debug application spamfilter 2

Fortinet support will advise you about which debugging level you should use.

You can enable timestamps to the debug output, using the following command:

diagnose debug console timestamp enable

When you finish examining the debug output, disable it, using the following command:

diagnose debug disable

Debug output example

This example shows the IKE negotiation for a secure logging connection from a FortiGate to a FortiAnalyzer.

diagnose debug reset

diagnose vpn ike log-filter src-addr4 192.168.11.2

diagnose debug enable

Sample output:

FGh_FtiLog1: IPsec SA connect 0 192.168.11.2->192.168.10.201:500, nat_mode=0 rekey=0 phase2=FGh_FtiLog1

FGh_FtiLog1: using existing connection, dpd_fail=0

FGh_FtiLog1: found phase2 FGh_FtiLog1

FGh_FtiLog1: IPsec SA connect 0 192.168.11.2 -> 192.168.10.201:500 negotiating

FGh_FtiLog1: overriding selector 225.30.5.8 with 192.168.11.2

FGh_FtiLog1: initiator quick-mode set pfs=1536...

FGh_FtiLog1: try to negotiate with 1800 life seconds.

FGh_FtiLog1: initiate an SA with selectors: 192.168.11.2/0.0.0.0->192.168.10.201, ports=0/0, protocol=0/0

Send IKE Packet(quick_outI1):192.168.11.2:500(if0) -> 192.168.10.201:500, len=348

Initiator: sent 192.168.10.201 quick mode message #1 (OK)

FGh_FtiLog1: set retransmit: st=168, timeout=6.

In this example:

192.168.11.2->192.168.10.201:500

Source and destination gateway IP address

dpd_fail=0

Found existing Phase 1

pfs=1536...

Create new Phase 2 tunnel

The execute tac report command

exec tac report is an execute command that runs an exhaustive series of diagnostic commands. It runs commands that are only needed if you're using certain features, such as HA, VPN tunnels, or a modem. The report takes a few minutes to finish because of the amount of output that's generated. If you're logging CLI output to a file, you can run this command to familiarize yourself with the diagnostic commands.

When you contact Fortinet Support, you may be asked to use the output from this CLI command to provide information about your FortiGate and its current state.

Other commands

ARP table

To view the ARP cache, use the following command:

get system arp

To view the ARP cache in the system, use the following command:

diagnose ip arp list

Sample output:

index=14 ifname=internal 224.0.0.5 01:00:5e:00:00:05 state=00000040 use=72203 confirm=78203 update=72203 ref=1

index=13 ifname=dmz 192.168.3.100 state=00000020 use=1843 confirm=650179 update=644179 ref=2 ? VIP

index=13 ifname=dmz 192.168.3.109 02:09:0f:78:69:ff state=00000004 use=71743 confirm=75743 update=75743 ref=1

index=14 ifname=internal 192.168.11.56 00:1c:23:10:f8:20 state=00000004 use=10532 confirm=10532 update=12658 ref=4

To remove the ARP cache, use the following command:

execute clear system arp table

To remove a single ARP entry, use the following command:

diagnose ip arp delete <interface name> <IP address>

To add static ARP entries, use the following command:

config system arp-table

Time and date settings

Check time and date settings for log message timestamp synchronization (the Fortinet support group may request this) and for certificates that have a time requirement to check for validity. Use the following commands:

execute time

current time is: 12:40:48

last ntp sync:Thu Mar 16 12:00:21 2006

execute date

current date is: 2006-03-16

To force synchronization with an NTP server, use the following command:

config system ntp

set ntpsync {enable|disable}

end

If all devices have the same time, it helps to correlate log entries from different devices.

IP address

There may be times when you want to verify that the IP addresses assigned to the FortiGate interfaces are what you expect them to be. This is easily accomplished from the CLI, using the following command:

diagnose ip address list

The output from this command lists the IP address and mask (if available), the indexof the interface (a type of ID number), and the devname (the interface name). While physical interface names are set, virtual interface names can vary. A good way to use this command is to list all of the virtual interface names. Listing all the virtual interface names is a good use of this command. For vsys_ha and vsys_fgfm, the IP addresses are the local host, which are virtual interfaces that are used internally.

# diagnose ip address list

IP=10.31.101.100->10.31.101.100/255.255.255.0 index=3 devname=internal

IP=172.20.120.122->172.20.120.122/255.255.255.0 index=5 devname=wan1

IP=127.0.0.1->127.0.0.1/255.0.0.0 index=8 devname=root

IP=127.0.0.1->127.0.0.1/255.0.0.0 index=11 devname=vsys_ha

IP=127.0.0.1->127.0.0.1/255.0.0.0 index=13 devname=vsys_fgfm

FortiOS diagnostics

FortiOS has a collection of diagnostic commands that you can use to troubleshoot and monitor the performance of your network. The get and diagnose CLI commands are the two main groups of diagnostic commands. Both commands display information about system resources, connections, and settings that allow you to locate problems or monitor system performance.

This topic includes diagnostics commands to help with:

Date and time

The system date and time are important for FortiGuard services, logging events, and sending alerts. The wrong time makes the log entries confusing and difficult to use.

Use Network Time Protocol (NTP) to set the date and time, if possible. This is an automatic method that doesn't require manual intervention. However, you must ensure that the port is allowed through the firewalls on your network. FortiToken synchronization requires NTP in many situations.

How to set the date and time - GUI
  1. Go to the System Information widget on the Dashboard. The date and time are displayed next to System Time.
  2. To adjust the date and time settings, go to System > Settings.
  3. In System Time, you can set the time zone, date and time, and select NTP usage.
How to check the date and time - CLI

You can check the date and time using the CLI commands execute date and execute time.

config system global

set timezone <integer>

end

config system ntp

set type custom

config ntpserver

edit 1

set server “ntp1.fortinet.net”

next

edit 2

set server “ntp2.fortinet.net”

next

end

set ntpsync enable

set syncinterval 60

end

Use the set timezone ? command to display a list of timezones and the integers that represent them.

Resource usage

Each program that runs on a computer has one or more processes associated with it. For example, if you open a Telnet program, it has an associated Telnet process. The same is true in FortiOS. All processes share the system resources in FortiOS, including memory and CPU.

Use the get system performance status command to show the FortiOS performance status.

Sample output:

FGT# get system performance status

CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

Memory: 4050332k total, 527148k used (13%), 3381312k free (83%), 141872k freeable (3%)

Average network usage: 41 / 28 kbps in 1 minute, 54 / 44 kbps in 10 minutes, 42 / 34 kbps in 30 minutes

Average sessions: 33 sessions in 1 minute, 48 sessions in 10 minutes, 38 sessions in 30 minutes

Average session setup rate: 0 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes

Virus caught: 0 total in 1 minute

IPS attacks blocked: 0 total in 1 minute

Uptime: 0 days, 22 hours, 59 minutes

Monitor the CPU and memory usage of internal processes, using the following command:

get system performance top <delay> <max_lines>

The data that the command lists includes the name of the daemon, the process ID, whether the process is sleeping or running, the CPU use percentage, and the memory use percentage.

Sample output:

get system performance top 10 100

Run Time: 0 days, 23 hours and 4 minutes

0U, 0N, 0S, 100I, 0WA, 0HI, 0SI, 0ST; 3955T, 3298F

httpsd 212 S 0.4 0.6

forticron 169 S 0.4 0.4

newcli 4054 R 0.4 0.2

reportd 174 S 0.0 1.4

pyfcgid 325 S 0.0 0.8

cmdbsvr 141 S 0.0 0.7

miglogd 160 S 0.0 0.6

httpsd 211 S 0.0 0.6

src-vis 180 S 0.0 0.6

pyfcgid 327 S 0.0 0.6

pyfcgid 328 S 0.0 0.6

pyfcgid 329 S 0.0 0.6

httpsd 162 S 0.0 0.5

cw_acd 189 S 0.0 0.5

httpsd 3998 S 0.0 0.5

httpsd 4050 S 0.0 0.5

updated 176 S 0.0 0.5

httpsd 4052 S 0.0 0.4

miglogd 203 S 0.0 0.4

miglogd 204 S 0.0 0.4

Proxy operation

Monitor proxy operations, using the following command:

diagnose test application <application> <option>

o display a list of available <application> values, enter:

diagnose test application ?

The <option> value depends on the application value that you use in the command. To display a list of available <option> values, enter:

diagnose test application <application> ?

For example, if the application is http, the CLI command that displays the <option> values is:

diagnose test application http ?

Hardware NIC

To monitor hardware network operations, use the following command:

diagnose hardware deviceinfo nic <interface>

The information that this command shows is important because errors at the interface indicate data link or physical layer issues which may impact the performance of the FortiGate.

The following example shows a sample output when you set <interface> to lan:

System_Device_Name lan

Current_HWaddr 00:09:0f:68:35:60

Permanent_HWaddr 00:09:0f:68:35:60

State up

Link up

Speed 100

Duplex full

[……]

Rx_Packets=5685708

Tx_Packets=4107073

Rx_Bytes=617908014

Tx_Bytes=1269751248

Rx_Errors=0

Tx_Errors=0

Rx_Dropped=0

Tx_Dropped=0

[……]

The diagnose hardware deviceinfo nic command displays a list of error names and values that are related to hardware. The following table describes possible hardware errors:

Field

Description

Rx_Errors = rx error count

Bad frame was marked as error by PHY

Rx_CRC_Errors +

Rx_Length_Errors -

Rx_Align_Errors

This error is only valid in 10/100M mode

Rx_Dropped or

Rx_No_Buffer_Count

Running out of buffer space

Rx_Missed_Errors

Equals Rx_FIFO_Errors + CEXTERR (Carrier Extension Error Count); only valid in 1000M mode, which is marked by PHY

Tx_Errors = Tx_Aborted_Errors

ECOL (Excessive Collisions Count); only valid in half-duplex mode

Tx_Window_Errors

Late Collisions (LATECOL) Count

Late collisions are collisions that occur after 64-byte time into the transmission of the packet while working in 10 to 100 Mb/s data rate and 512-byte time into the transmission of the packet while working in the 1,000 Mb/s data rate. This register only increments if transmits are enabled and the device is in half-duplex mode.

Rx_Dropped

See Rx_Errors

Tx_Dropped

Not defined

Collisions

Total number of collisions experienced by the transmitter; valid in half-duplex mode

Rx_Length_Errors

Transmission length error

Rx_Over_Errors

Not defined

Rx_CRC_Errors

Frame CRC error

Rx_Frame_Errors

Same as Rx_Align_Errors

This error is only valid in 10/100M mode.

Rx_FIFO_Errors

Same as Rx_Missed_Errors - a missed packet count

Tx_Aborted_Errors

See Tx_Errors

Tx_Carrier_Errors

The PHY should assert the internal carrier sense signal during every transmission. Failure to do so may indicate that the link has failed or the PHY has an incorrect link configuration. This register only increments if transmits are enabled. This register isn't valid in internal SerDes 1 mode (TBI mode for the 82544GC/EI) and is valid only when the Ethernet controller is operating at full duplex.

Tx_FIFO_Errors

Not defined

Tx_Heartbeat_Errors

Not defined

Tx_Window_Errors

See LATECOL

Tx_Single_Collision_Frames

Counts the number of times that a successfully transmitted packet encountered a single collision

The value increments only if transmits are enabled and the Ethernet controller is in half-duplex mode.

Tx_Multiple_Collision_Frames

A Multiple Collision Count which indicates the number of times that a transmit encountered more than one collision, but less than 16. The value increments only if transmits are enabled and the Ethernet controller is in half-duplex mode.

Tx_Deferred

Counts defer events

A deferred event occurs when the transmitter can't immediately send a packet due to the medium being busy because another device is transmitting, the IPG timer hasn't expired, half-duplex deferral events are occurring, XOFF frames are being received, or the link isn't up. This register only increments if transmits are enabled. This counter doesn't increment for streaming transmits that are deferred due to TX IPG.

Rx_Frame_Too_Longs

The Rx frame is oversized

Rx_Frame_Too_Shorts

The Rx frame is too short

Rx_Align_Errors

This error is only valid in 10/100M mode

Symbol Error Count

Counts the number of symbol errors between reads - SYMERRS.

The count increases for every bad symbol that's received, whether or not a packet is currently being received and whether or not the link is up. This register increments only in internal SerDes mode.

Traffic trace

Traffic tracing allows you to follow a specific packet stream. This is useful to confirm that packets are taking the route you expect them to take on your network.

View the characteristics of a traffic session though specific security policies using:

diagnose sys session

Trace per-packet operations for flow tracing using:

diagnose debug flow

Trace per-Ethernet frame using:

diagnose sniffer packet

Trace a route from a FortiGate to a destination IP address using:

# execute traceroute www.fortinet.com

traceroute to www.fortinet.com (66.171.121.34), 32 hops max, 84 byte packets

1 172.20.120.2 0.637 ms 0.653 ms 0.279 ms

2 209.87.254.221 <static-209-87-254-221.storm.ca> 2.448 ms 2.519 ms 2.458 ms

3 209.87.239.129 <core-2-g0-2.storm.ca> 2.917 ms 2.828 ms 9.324 ms

4 209.87.239.199 <core-3-bdi1739.storm.ca> 13.248 ms 12.401 ms 13.009 ms

5 216.66.41.113 <v502.core1.tor1.he.net> 17.181 ms 12.422 ms 12.268 ms

6 184.105.80.9 <100ge1-2.core1.nyc4.he.net> 21.355 ms 21.518 ms 21.597 ms

7 198.32.118.41 <ny-paix-gni.twgate.net> 83.297 ms 84.416 ms 83.782 ms

8 203.160.228.217 <217-228-160-203.TWGATE-IP.twgate.net> 82.579 ms 82.187 ms 82.066 ms

9 203.160.228.229 <229-228-160-203.TWGATE-IP.twgate.net> 82.055 ms 82.455 ms 81.808 ms

10 203.78.181.2 82.262 ms 81.572 ms 82.015 ms

11 203.78.186.70 83.283 ms 83.243 ms 83.293 ms

12 66.171.127.177 84.030 ms 84.229 ms 83.550 ms

13 66.171.121.34 <www.fortinet.com> 84.023 ms 83.903 ms 84.032 ms

14 66.171.121.34 <www.fortinet.com> 83.874 ms 84.084 ms 83.810 ms

Session table

A session is a communication channel between two devices or applications across the network. Sessions allow FortiOS to inspect and act on a sequential group of packets in a session all together instead of inspecting each packet individually. Each of these sessions has an entry in the session table that includes important information about the session.

Use as a tool

Session tables are useful troubleshooting tools because they allow you to verify open connections. For example, if you have a web browser open to browse the Fortinet website, you would expect a session entry from your computer, on port 80, to the IP address for the Fortinet website. Another troubleshooting method is if there are too many sessions for FortiOS to process, you can examine the session table for evidence why this is happening.

You can view the FortiGate session table from either the FortiGate GUI or CLI. The most useful troubleshooting data comes from the CLI. The session table in the GUI also provides useful summary information, particularly the current policy number that the session is using.

GUI session information

You can view session information by going to the FortiView page. Read more about FortiView consoles in the Handbook's FortiView chapter.

How to find which security policy a specific connection is using

Every program and device on your network must have a communication channel, or session, open to pass information. The FortiGate manages these sessions with features such as traffic shaping, antivirus scanning, and blocking known bad web sites. Each session has an entry in the session table.

You may want to find information for a specific session for troubleshooting. For example, if a secure web browser session isn't working properly, you can check the session table to ensure the session is still active and going to the proper address. The session table can also tell you the security policy number it matches, so you can check what's happening in that policy.

  1. Know your connection information.

    You need to be able to identify the session you want. For this, you need the source IP address (usually your computer), the destination IP address (if you have it), and the port number which is determined by the program that you're using. Some commons ports are:

    • Port 80 (HTTP for web browsing)
    • Port 80 (HTTP for web browsing)
    • Port 80 (HTTP for web browsing)
    • Port 80 (HTTP for web browsing)
  2. Find your session and policy ID.

    Go to FortiView > All Sessions. Find your session by finding your source IP address, destination IP address (if you have it), and port number. The policy ID is listed after the destination information. If the list of sessions is very long, you can filter the list to make it easier to find your session.

  3. When there are many sessions, use a filter to help you find your session.

    If there are multiple pages of sessions, it's difficult to find a single session. You can use a filter to block out sessions that you don’t want. Click the search icon on the column heading to select the filter. Select Source IP and enter your source IP address. Now, only sessions that originate from your IP address are displayed in the session table. If the list is still too long, you can do the same for the Source port. That makes it easy to find your session and the security policy ID.

CLI session information

The session table output that the diagnose sys session list command generates is very large. You can use filters to display only the session data that you're interested in.

An entry is placed in the session table for each traffic session passing through a security policy. The following command lists the information for a session in the table:

diagnose sys session list

The filter option displays specific information, for example:

diagnose sys session filter <option>

The values for <option> include the following:

clear

Clear session filter

dintf

Destination interface

dport

Destination port

dst

Destination IP address

duration

Duration of the session

expire

Expire

negate

Inverse filter

nport

NAT'd source port

nsrc

NAT'd source ip address

policy

Policy ID

proto

Protocol number

proto-state

Protocol state

session-state1

Session state1

session-state2

Session state2

sintf

Source interface

sport

Source port

src

Source IP address

vd

Index of virtual domain, -1 matches all

Even though UDP is a sessionless protocol, the FortiGate keeps track of the following states:

  • When UDP reply doesn't have a value of 0
  • When UDP reply has a value of 1

The following table displays firewall session states from the session table:

State

Description

log

Session is being logged

local

Session is originated from or destined for local stack

ext

Session is created by a firewall session helper

may_dirty

Session is created by a policy

For example, the session for ftp control channel will have this state but ftp data channel won't. This is also seen when NAT is enabled.

ndr

Session will be checked by IPS signature

nds

Session will be checked by IPS anomaly

br

Session is being bridged (TP) mode

Firewall session setup rate

The number of sessions that can be established in a set period of time is useful information. A session is an end-to-end TCP/IP connection for communication with a limited lifespan. If you record the setup rate during normal operation, when you experience problems you can compare the baseline setup rate to the rate that occurs when you're troubleshooting. This can be a useful step to help you define your problem.

A reduced firewall session setup rate can be the result of a number of things, such as a lack of system resources on the FortiGate or reaching the limit of your session count for your VDOM.

To view your session setup rate method 1 - CLI

FGT# get system performance status

CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

Memory: 4050332k total, 530512k used (13%), 3376844k free (83%), 142976k freeable (3%)

Average network usage: 131 / 90 kbps in 1 minute, 26 / 15 kbps in 10 minutes, 49 / 42 kbps in 30 minutes

Average sessions: 80 sessions in 1 minute, 30 sessions in 10 minutes, 42 sessions in 30 minutes

Average session setup rate: 3 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes

Virus caught: 0 total in 1 minute

IPS attacks blocked: 0 total in 1 minute

Uptime: 1 days, 2 hours, 45 minutes

The information you're looking for is the average sessions section, in the above output. This example shows that there were 80 sessions in 1 minute, or an average of 3 sessions per second. The values for 10 minutes and 30 minutes allow you to take a longer average for a more reliable value if your FortiGate is working at maximum capacity. The smallest FortiGate can have 1,000 sessions established per second across the unit.

Remember that session setup rate is a global command. If you have multiple VDOMs configured with many sessions in each one, the session setup rate per VDOM will be slower than if there are no VDOMs configured.

Finding object dependencies

An administrator may not be permitted to delete a configuration object if there are other configuration objects that depend on it. This command identifies other objects which depend on, or make reference to, the configuration object in question. If an error is displayed that an object is in use and can't be deleted, this command can help identify the source of the problem.

Additionally, if you have a virtual interface with objects that depend on it, you need to find and remove those dependencies before you delete the interface.

CLI method

When you run multiple VDOMs, you use this command in the global configuration only and it searches for the named object in both the most recently used global and VDOM configurations:

diagnose sys cmdb refcnt show <path.object.mkey> <tablename>

For example, to verify which objects a security policy with an ID of 1 refers to, enter the following command:

diagnose sys cmdb refcnt show firewall.policy.policyid 1

To check what is referred to by interface port1, enter the following command:

diagnose sys cmdb refcnt show system.interface.name port1

To show all dependencies for an interface, enter the following command:

diagnose sys cmdb refcnt show system.interface.name <interface name>

Sample output:

entry used by table firewall.address:name '10.98.23.23_host’

entry used by table firewall.address:name 'NAS'

entry used by table firewall.address:name 'all'

entry used by table firewall.address:name 'fortinet.com'

entry used by table firewall.vip:name 'TORRENT_10.0.0.70:6883'

entry used by table firewall.policy:policyid '21'

entry used by table firewall.policy:policyid '14'

entry used by table firewall.policy:policyid '19'

In this example, the interface has dependent objects, including four address objects, one VIP, and three security policies.

GUI method

In the GUI, you can easily check and remove the object dependencies for an interface.

To remove interface object dependencies - GUI
  1. Go to Network > Interfaces. The Ref. column displays the number of objects that refer to this interface.
  2. Select the number in the Ref. column for the interface. A window listing the dependencies appears.
  3. Use these detailed entries to locate and remove object references to this interface. The trash can icon changes from gray when you've removed all object dependencies.
  4. Remove the interface by selecting the check box for the interface, and select Delete.

Flow trace

To trace the flow of packets through the FortiGate, use the following command:

diagnose debug flow trace start

Follow packet flow by setting a flow filter, using this command:

diagnose debug flow {filter | filter6} <option>

If your network uses IPv4, enter filter. If your network uses IPv6, enter filter6.

One of the following variables replaces <option>:

Variable

Description

addr

IPv4 or IPv6 address

clear

clear filter

daddr

destination IPv4 or IPv6 address

dport

destination port

negate

inverse IPv4 or IPv6 filter

port

port

proto

protocol number

saddr

source address

sport

source port

vd

index of virtual domain; -1 matches all

caution icon

diagnose debug flow output is recorded as event log messages and which are then sent to a FortiCloud or a FortiAnalyzer, if connected. Don't run this command longer than necessary, since it generates significant amounts of data.

To start flow monitoring with a specific number of packets - CLI:

diagnose debug flow trace start <N>

To stop flow tracing at any time using - CLI:

diagnose debug flow trace stop

The following example shows the flow trace for a device with an IP address of 203.160.224.97:

diagnose debug enable

diagnose debug flow filter addr 203.160.224.97

diagnose debug flow show function-name enable

diagnose debug flow trace start 100

Flow trace output example - HTTP

To observe the debug flow trace, connect to the web site at the following address:

https://www.fortinet.com

Comment: SYN packet received:

id=20085 trace_id=209 func=resolve_ip_tuple_fast

line=2700 msg="vd-root received a packet(proto=6,

192.168.3.221:1487->203.160.224.97:80) from port5."

SYN sent and a new session is allocated:

id=20085 trace_id=209 func=resolve_ip_tuple line=2799

msg="allocate a new session-00000e90"

Lookup for next-hop gateway address:

id=20085 trace_id=209 func=vf_ip4_route_input line=1543

msg="find a route: gw-192.168.11.254 via port6"

Source NAT, lookup next available port:

id=20085 trace_id=209 func=get_new_addr line=1219

msg="find SNAT: IP-192.168.11.59, port-31925"

direction“

Matched security policy. Check to see which policy this session matches:

id=20085 trace_id=209 func=fw_forward_handler line=317

msg="Allowed by Policy-3: SNAT"

Apply source NAT:

id=20085 trace_id=209 func=__ip_session_run_tuple

line=1502 msg="SNAT 192.168.3.221->192.168.11.59:31925"

SYN ACK received:

id=20085 trace_id=210 func=resolve_ip_tuple_fast line=2700

msg="vd-root received a packet(proto=6, 203.160.224.97:80-

>192.168.11.59:31925) from port6."

Found existing session ID. Identified as the reply direction:

id=20085 trace_id=210 func=resolve_ip_tuple_fast line=2727

msg="Find an existing session, id-00000e90, reply direction"

Apply destination NAT to inverse source NAT action:

id=20085 trace_id=210 func=__ip_session_run_tuple

line=1516 msg="DNAT 192.168.11.59:31925-

>192.168.3.221:1487"

Lookup for next-hop gateway address for reply traffic:

id=20085 trace_id=210 func=vf_ip4_route_input line=1543

msg="find a route: gw-192.168.3.221 via port5"

ACK received:

id=20085 trace_id=211 func=resolve_ip_tuple_fast line=2700

msg="vd-root received a packet(proto=6,

192.168.3.221:1487->203.160.224.97:80) from port5."

Match existing session in the original direction:

id=20085 trace_id=211 func=resolve_ip_tuple_fast line=2727

msg="Find an existing session, id-00000e90, original

direction"

Apply source NAT:

id=20085 trace_id=211 func=__ip_session_run_tuple

line=1502 msg="SNAT 192.168.3.221->192.168.11.59:31925"

Receive data from client:

id=20085 trace_id=212 func=resolve_ip_tuple_fast

line=2700 msg="vd-root received a packet(proto=6,

192.168.3.221:1487->203.160.224.97:80) from port5."

Match existing session in the original direction:

id=20085 trace_id=212 func=resolve_ip_tuple_fast

line=2727 msg="Find an existing session, id-00000e90,

original direction"

Apply source NAT:

id=20085 trace_id=212 func=__ip_session_run_tuple

line=1502 msg="SNAT 192.168.3.221->192.168.11.59:31925"

Receive data from server:

id=20085 trace_id=213 func=resolve_ip_tuple_fast

line=2700 msg="vd-root received a packet(proto=6,

203.160.224.97:80->192.168.11.59:31925) from port6."

Match existing session in reply direction:

id=20085 trace_id=213 func=resolve_ip_tuple_fast

line=2727 msg="Find an existing session, id-00000e90,

reply direction"

Apply destination NAT to inverse source NAT action:

id=20085 trace_id=213 func=__ip_session_run_tuple

line=1516 msg="DNAT 192.168.11.59:31925-

>192.168.3.221:1487"

Flow trace output example - IPsec (policy-based)

id=20085 trace_id=1 msg="vd-root received a packet(proto=1, 10.72.55.240:1->10.71.55.10:8) from internal."

id=20085 trace_id=1 msg="allocate a new session-00001cd3"

id=20085 trace_id=1 msg="find a route: gw-66.236.56.230 via wan1"

id=20085 trace_id=1 msg="Allowed by Policy-2: encrypt"

id=20085 trace_id=1 msg="enter IPsec tunnel-RemotePhase1"

id=20085 trace_id=1 msg="encrypted, and send to 15.215.225.22 with source 66.236.56.226"

id=20085 trace_id=1 msg="send to 66.236.56.230 via intf-wan1“

id=20085 trace_id=2 msg="vd-root received a packet (proto=1, 10.72.55.240:1-1071.55.10:8) from internal."

id=20085 trace_id=2 msg="Find an existing session, id-00001cd3, original direction"

id=20085 trace_id=2 msg="enter IPsec ="encrypted, and send to 15.215.225.22 with source 66.236.56.226“ tunnel-RemotePhase1"

id=20085 trace_id=2 msgid=20085 trace_id=2 msg="send to 66.236.56.230 via intf-wan1"

Packet sniffing and packet capture

When you troubleshoot networks, it helps to look inside the header of the packets. This helps to determine if the packets, route, and destination are all what you expect. Packet capture can also be called a network tap, packet sniffing, or logic analyzing. FortiOS devices can sniff packets using CLI commands or capture packets using the GUI.

Packet sniffing using CLI commands is well-suited for spot checking traffic, but if you have complex filters to enter it can be a lot of work to enter them each time. You can also save the sniffing output. However, you must log to a file and then analyze the file later.

Packet capture in the GUI makes it easy for you to set up multiple filters at once and run only one or two as you need to. You can also use controls to start and stop capturing when you want to. You download packet capture output to your local computer as a *.pcap file. You must use a third party application, such as Wireshark, to read *.pcap files. This method is useful to send information to Fortinet support to help resolve an issue.

The following table presents a comparison between the two methods:

Features

Packet sniffing

Packet capture

Command location

CLI

GUI

Third party software required

puTTY to log plaintext output

Wireshark, or similar application, to read *.pcap files

Read output in plain text file

yes

no

Read output as *.pcap file using Wireshark, or similar application

no

yes

Easily configure single quick and simple filter

yes

no

Record packet interface

yes

no

Configure complex sniffer filters on multiple interface

no

yes

sniff IPv6

hard

easy

sniff non-IP packets

no

yes

Filter packets by protocol and/or port

easy

easy

Filter packets by source and/or destination address

easy

easy

Packet sniffing

If you're running a constant traffic application, such as ping, packet sniffing can tell you if the traffic is reaching the destination, what the port of entry is on the FortiGate unit, if the ARP resolution is correct, and if the traffic is being sent back to the source as expected.

Sniffing packets can also tell you if the FortiGate is silently dropping packets for reasons such as Reverse Path Forwarding (RPF). RPF, also called anti-spoofing, prevents an IP packet from being forwarded if its source IP doesn't belong to a locally attached subnet (local interface) or isn't part of the routing between the FortiGate and another source (static route, RIP, OSPF, BGP). Note that you can disable RPF by turning on asymmetric routing in the CLI (config system settings, set asymroute enable), but this disables stateful inspection on the FortiGate and causes many features to be turned off.

caution icon

If you configure virtual IP addresses on your FortiGate, it will use those addresses instead of the physical IP addresses. You'll notice this when you're sniffing packets because all the traffic uses the virtual IP addresses. This is due to the ARP update that's sent out when the VIP address is configured.

Before you start sniffing packets on the CLI, you should prepare to capture the output to a file. A large amount of data may scroll by and you won't be able to see it without first saving it to a file. One method is to use a terminal program like puTTY to connect to the FortiGate CLI. Once the packet sniffing count is reached, you can end the session and analyze the output in the file.

The general form of the internal FortiOS packet sniffer command is:

diagnose sniffer packet <interface_name> <‘filter’> <verbose> <count> <tsformat>

To stop the sniffer, type CTRL+C.

<interface_name>

The name of the interface to sniff, such as “port1” or “internal”. This can also be “any” to sniff all interfaces.

<‘filter’>

What to look for in the information the sniffer reads. “none” indicates no filtering, and all packets are displayed as the other arguments indicate.

The filter must be inside single quotes (‘).

<verbose>

The level of verbosity as one of:

1 - print header of packets
2 - print header and data from IP of packets
3 - print header and data from Ethernet of packets
4 - print header of packets with interface name

<count>

The number of packets the sniffer reads before stopping. If you don't put a number here, the sniffer will run until you stop it with <CTRL+C>.

<tsformat>

The format of timestamp.
  • a: absolute UTC time, yyyy-mm-dd hh:mm:ss.ms
  • l: absolute LOCAL time, yyyy-mm-dd hh:mm:ss.ms
  • otherwise: relative to the start of sniffing, ss.ms

For a simple sniffing example, enter the CLI command diagnose sniffer packet port1 none 1 3. This displays the next three packets on the port1 interface using no filtering, and verbose level 1. At this verbosity level, you can see the source IP and port, the destination IP and port, action (such as ack), and sequence numbers.

In the output below, port 443 indicates these are HTTPS packets and that 172.20.120.17 is both sending and receiving traffic.

Head_Office_620b # diagnose sniffer packet port1 none 1 3

interfaces=[port1]

filters=[none]

0.545306 172.20.120.17.52989 -> 172.20.120.141.443: psh 3177924955 ack 1854307757

0.545963 172.20.120.141.443 -> 172.20.120.17.52989: psh 1854307757 ack 3177925808

0.562409 172.20.120.17.52988 -> 172.20.120.141.443: psh 4225311614 ack 3314279933

For a more advanced example of packet sniffing, the following commands will report packets on any interface that are traveling between a computer with the host name of “PC1” and a computer with the host name of “PC2”. With verbosity 4 and above, the sniffer trace displays the interface names where traffic enters or leaves the FortiGate unit. Remember to stop the sniffer, type CTRL+C.

FGT# diagnose sniffer packet any "host <PC1> or host <PC2>" 4

or

FGT# diagnose sniffer packet any "(host <PC1> or host <PC2>) and icmp" 4

The following CLI command for a sniffer includes the ARP protocol in the filter which may be useful to troubleshoot a failure in the ARP resolution (for example, PC2 may be down and not responding to the FortiGate ARP requests).

FGT# diagnose sniffer packet any "host <PC1> or host <PC2> or arp" 4

Packet capture

Packet capture tells you what is happening on the network at a low level. This can be very useful for troubleshooting problems, such as:

  • Finding missing traffic
  • Seeing if sessions are setting up properly
  • Locating ARP problems such as broadcast storm sources and causes
  • Confirming which address a computer is using on the network, if they have multiple addresses or are on multiple networks
  • Confirming routing is working as you expect
  • Connecting wireless clients
  • Missing PING packets
  • A particular type of packet is having problems, such as UDP, which is commonly used for streaming video

If you're running a constant traffic application such as ping, packet capture can tell you if the traffic is reaching the destination, how the port enters and exits the FortiGate, if the ARP resolution is correct, and if the traffic is returning to the source as expected. You can also use packet switching to verify that NAT or other configuration is translating addresses or routing traffic the way that you want it to.

Before you start capturing packets, you need to have a good idea of what you're looking for. Capture is used to confirm or deny your ideas about what's happening on the network. If you try capture without a plan to narrow your search, you can end up with too much data to effectively analyze. On the other hand, you need to capture enough packets to really understand all of the patterns and behavior that you're examining.

To use packet capture, the FortiGate must have a disk. You can enable the capture-packet in the firewall policy, using the following CLI commands:

config firewall policy

edit <id>

set capture-packet enable

end

To configure packet capture filters, go to Network > Packet Capture.

When you add a packet capture filter, enter the following information and select OK.

Interface

Select the interface to sniff from the drop-down menu.

You must select one interface. You can't change the interface without deleting the filter and creating a new one, unlike the other fields.

Max Packets to Save

Enter the number of packets to capture before the filter stops.

This number can't be zero. You can halt the capturing before this number is reached.

Enable Filters

Select this option to specify filter fields

Host(s)

Enter the IP address of one or more hosts.

Separate multiple hosts with commas. To enter a range, use a dash without spaces, for example 172.16.1.5-172.16.1.15, or enter a subnet.

Port(s)

Enter one or more ports to capture on the selected interface.

Separate multiple ports with commas. To enter a range, use a dash without spaces, for example 88-90

VLAN(s)

Enter one or more VLANs (if any). Separate multiple VLANs with commas.

Protocol

Enter one or more protocols. Separate multiple protocols with commas. To enter a range, use a dash without spaces, for example 1-6, 17, 21-25.

Include IPv6 Packets

Select this option if you're troubleshooting IPv6 networking, or if your network uses IPv6. Otherwise, leave it disabled.

Include Non-IP Packets

The protocols in the list are all IP based except for ICMP (ping). To capture non-IP based packets, select this feature. Examples of non-IP packets include IPsec, IGMP, ARP, and ICMP.

If you select a filter, you have the option to start and stop packet capture in the edit window, or download the captured packets. You can also see the filter status and the number of packets captured.

You can select the filter and start capturing packets. When the filter is running, the number of captured packets increases until it reaches the Max Packet Count or you stop it. When the filter is running, you can't download the output file.

When the packet capture is complete, you can download the *.pcap file. You must use a third party application, such as Wireshark, to read *,pcap files. This tool provides you with extensive analytics and the full contents of the packets that were captured.

To start, stop, or resume packet capture, use the symbols on the screen. These symbols are the same as those used for audio or video playback. Hover over the symbol to reveal explanatory text. Similarly, to download the *.pcap file, use the download symbol on the screen.

NPU-based interfaces

Many Fortinet products contain network processors, such as NP1, NP2, NP4, and NP6, which means that offloading requirements vary depending on the model.

When you use the NPU-based interfaces, you can see only the initial session setup will be seen using the diagnose debug flow command. If the session is correctly programmed into the ASIC (fastpath), the diagnose debug flow command won't detect the packets that arrive at the CPU. If the NPU functionality is disabled, the CPU detects all the packets. However, you should only this for troubleshooting purposes.

First, obtain the NP4 or NP6 ID and the port numbers, using the following command:

diagnose npu {np4|npu6}list

Sample output:

ID Model Slot Interface

0 On-board port1 fabric1 fabric3 fabric5

1 On-board fabric2 port2 base2 fabric4

Run the following commands:

diagnose npu {np4|npu6}fastpath disable <dev_id>

(where dev_id is the NP4 or NP6 number)

Then, run this command:

diagnose npu {np4|npu6}fastpath-sniffer enable port1

Sample output:

NP4 Fast Path Sniffer on port1 enabled

This causes traffic on port1 of the network processor to be sent to the CPU. this means that you can take a standard sniffer trace and use other diagnose commands, if it's a standard CPU-driven port.

These commands only apply to the newer NP4 and NP6 interfaces.

Debug command

Debug output provides continuous, real-time event information and continues until you stop it or reboot the unit. Debug output can affect system performance and is continually generated even though output might not be displayed in the CLI console.

Debug information that's displayed in the console scrolls in the console display and may prevent you from entering CLI commands, such as, the command to disable the debug display. To turn off debug output as the display scrolls by, press the key to recall the recent diagnose debug command, press backspace, and type “0”, and Enter.

To enable debug output display, use the following command:

diagnose debug enable

Once you enable debug output display, specify the debug information that you require using the following command:

diagnose debug <option> <level>

Debug command options include the following:

enable

Enable debug output

disable

Disable debug output

info

Show active debug level settings

reset

Reset all debug level to default

report

Report for tech support

crashlog

Crash log info

config-error-log

Configure error log info

sql-log-error

SQL log database error info

application

application

kernel

kernel

remote-extender

remote-extender

cli

Debug CLI

cmdb-trace

Trace CLI

rating

Display rating info

authd

Authentication daemon

fsso-polling

FSSO active directory poll module

flow

Trace packet flow in kernel

urlfilter

urlfilter

admin

Admin user

You can set the debug level at the end of the command. For example, typical values are 2 and 3, like in the following commands:

diagnose debug application DHCPS 2

diagnose debug application spamfilter 2

Fortinet support will advise you about which debugging level you should use.

You can enable timestamps to the debug output, using the following command:

diagnose debug console timestamp enable

When you finish examining the debug output, disable it, using the following command:

diagnose debug disable

Debug output example

This example shows the IKE negotiation for a secure logging connection from a FortiGate to a FortiAnalyzer.

diagnose debug reset

diagnose vpn ike log-filter src-addr4 192.168.11.2

diagnose debug enable

Sample output:

FGh_FtiLog1: IPsec SA connect 0 192.168.11.2->192.168.10.201:500, nat_mode=0 rekey=0 phase2=FGh_FtiLog1

FGh_FtiLog1: using existing connection, dpd_fail=0

FGh_FtiLog1: found phase2 FGh_FtiLog1

FGh_FtiLog1: IPsec SA connect 0 192.168.11.2 -> 192.168.10.201:500 negotiating

FGh_FtiLog1: overriding selector 225.30.5.8 with 192.168.11.2

FGh_FtiLog1: initiator quick-mode set pfs=1536...

FGh_FtiLog1: try to negotiate with 1800 life seconds.

FGh_FtiLog1: initiate an SA with selectors: 192.168.11.2/0.0.0.0->192.168.10.201, ports=0/0, protocol=0/0

Send IKE Packet(quick_outI1):192.168.11.2:500(if0) -> 192.168.10.201:500, len=348

Initiator: sent 192.168.10.201 quick mode message #1 (OK)

FGh_FtiLog1: set retransmit: st=168, timeout=6.

In this example:

192.168.11.2->192.168.10.201:500

Source and destination gateway IP address

dpd_fail=0

Found existing Phase 1

pfs=1536...

Create new Phase 2 tunnel

The execute tac report command

exec tac report is an execute command that runs an exhaustive series of diagnostic commands. It runs commands that are only needed if you're using certain features, such as HA, VPN tunnels, or a modem. The report takes a few minutes to finish because of the amount of output that's generated. If you're logging CLI output to a file, you can run this command to familiarize yourself with the diagnostic commands.

When you contact Fortinet Support, you may be asked to use the output from this CLI command to provide information about your FortiGate and its current state.

Other commands

ARP table

To view the ARP cache, use the following command:

get system arp

To view the ARP cache in the system, use the following command:

diagnose ip arp list

Sample output:

index=14 ifname=internal 224.0.0.5 01:00:5e:00:00:05 state=00000040 use=72203 confirm=78203 update=72203 ref=1

index=13 ifname=dmz 192.168.3.100 state=00000020 use=1843 confirm=650179 update=644179 ref=2 ? VIP

index=13 ifname=dmz 192.168.3.109 02:09:0f:78:69:ff state=00000004 use=71743 confirm=75743 update=75743 ref=1

index=14 ifname=internal 192.168.11.56 00:1c:23:10:f8:20 state=00000004 use=10532 confirm=10532 update=12658 ref=4

To remove the ARP cache, use the following command:

execute clear system arp table

To remove a single ARP entry, use the following command:

diagnose ip arp delete <interface name> <IP address>

To add static ARP entries, use the following command:

config system arp-table

Time and date settings

Check time and date settings for log message timestamp synchronization (the Fortinet support group may request this) and for certificates that have a time requirement to check for validity. Use the following commands:

execute time

current time is: 12:40:48

last ntp sync:Thu Mar 16 12:00:21 2006

execute date

current date is: 2006-03-16

To force synchronization with an NTP server, use the following command:

config system ntp

set ntpsync {enable|disable}

end

If all devices have the same time, it helps to correlate log entries from different devices.

IP address

There may be times when you want to verify that the IP addresses assigned to the FortiGate interfaces are what you expect them to be. This is easily accomplished from the CLI, using the following command:

diagnose ip address list

The output from this command lists the IP address and mask (if available), the indexof the interface (a type of ID number), and the devname (the interface name). While physical interface names are set, virtual interface names can vary. A good way to use this command is to list all of the virtual interface names. Listing all the virtual interface names is a good use of this command. For vsys_ha and vsys_fgfm, the IP addresses are the local host, which are virtual interfaces that are used internally.

# diagnose ip address list

IP=10.31.101.100->10.31.101.100/255.255.255.0 index=3 devname=internal

IP=172.20.120.122->172.20.120.122/255.255.255.0 index=5 devname=wan1

IP=127.0.0.1->127.0.0.1/255.0.0.0 index=8 devname=root

IP=127.0.0.1->127.0.0.1/255.0.0.0 index=11 devname=vsys_ha

IP=127.0.0.1->127.0.0.1/255.0.0.0 index=13 devname=vsys_fgfm