Checking CPU and memory resources
Check the CPU and memory resources when the FortiGate is not working, the network is slow, or there is a reduced firewall session setup rate. All processes share the system resources in FortiOS, including CPU and memory.
To view system resources in the GUI:
Go to Dashboard > Status.
The resource information is located in the CPU and Memory widgets. For information, see Dashboard.
To view system resources in the CLI:
get system performance status
Sample output:
FGT# get system performance status
CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
Memory: 4050332k total, 527148k used (13%), 3381312k free (83%), 141872k freeable (3%)
Average network usage: 41 / 28 kbps in 1 minute, 54 / 44 kbps in 10 minutes, 42 / 34 kbps in 30 minutes
Average sessions: 33 sessions in 1 minute, 48 sessions in 10 minutes, 38 sessions in 30 minutes
Average session setup rate: 0 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes
Virus caught: 0 total in 1 minute
IPS attacks blocked: 0 total in 1 minute
Uptime: 0 days, 22 hours, 59 minutes
The first line of the output shows the CPU usage by category:
CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
The second line of the output shows the memory usage:
Memory: 4050332k total, 527148k used (13%), 3381312k free (83%), 141872k freeable (3%)
Memory usage should not exceed 90%. Using too much memory prevents some processes from functioning properly. For example, if the system is running low on memory, antivirus scanning enters into failopen mode where it drops connections or bypasses the antivirus system.
Other lines of output, such as average network usage, average session setup rate, viruses caught, and IPS attacks blocked, help determine why system resource usage is high.
For example:
- A high
average network usagemay indicate high traffic processing on the FortiGate, - A very low or zero,
average session setup ratemay indicate the proxy is overloaded and unable to do its job.
Troubleshooting CPU and network resources
FortiGate has stopped working
If the FortiGate has stopped working, the first line of the output will look similar to this:
CPU states: 0% user 0% system 0% nice 100% idle
Network is slow
If your network is running slow, the first line of the output will look similar to this:
CPU states: 1% user 98% system 0% nice 1% idle
This example shows that all of the CPU is being used by system processes, and the FortiGate is overloaded. When overloading occurs, it is possible a process such as scanunitid is using all the resources to scan traffic. In this case you need to reduce the amount of traffic being scanned by blocking unwanted protocols, configuring more security policies to limit scanning to certain protocols, or similar actions.
It is also possible a hacker has accessed your network and is overloading it with malicious activity, such as running a spam server or using zombie PCs to attack other networks on the Internet.
You can use the following command to investigate the problem with the CPU:
get system performance top
This command shows all of the top processes that are running on the FortiGate and their CPU usage. The process names are on the left. If a process is using most of the CPU cycles, investigate it to determine whether the activity is normal.
Reduced firewall session setup rate
A reduced firewall session setup rate can be caused by a lack of system resources on the FortiGate, or reaching the session count limit for a VDOM.
|
As a best practice, administrators should record the session setup rate during normal operation to establish a baseline to help define a problem when your are troubleshooting. |
The session setup rate appears in the average sessions section of the output.
A reduced firewall session setup rate will look similar to this:
Average sessions: 80 sessions in 1 minute, 30 sessions in 10 minutes, 42 sessions in 30 minutes
Average session setup rate: 3 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes
In the example above, there were 80 sessions in 1 minute, or an average of 3 sessions per second.
The values for 10 minutes and 30 minutes allow you to take a longer average for a more reliable value if your FortiGate is working at maximum capacity. The smallest FortiGate can have 1,000 sessions established per second across the unit.
|
The session setup rate is a global command. If you have multiple VDOMs configured with many sessions in each VDOM, the session setup rate per VDOM will be slower than if there are no VDOMs configured. |
High memory usage
As with any system, a FortiGate has limited hardware resources, such as memory, and all processes running on the FortiGate share the memory. Each process uses more or less memory, depending on its workload. For example, a process usually uses more memory in high traffic situations. If some processes use all of the available memory, other processes will not be able to run.
When high memory usage occurs, the services may freeze up, connections may be lost, or new connections may be refused.
If you see high memory usage in the Memory widget, the FotiGate may be handling high traffic volumes. Alternatively, the FortiGate may have problems with connection pool limits that are affecting a single proxy. If the FortiGate receives large volumes of traffic on a specific proxy, the unit may exceed the connection pool limit. If the number of free connections within a proxy connection pool reaches zero, issues may occur.
To view current memory usage information in the CLI:
diagnose hardware sysinfo memory
Sample output:
total: used: free: shared: buffers: cached: shm:
Mem: 2074185728 756936704 1317249024 0 20701184 194555904 161046528
Swap: 0 0 0
MemTotal: 2025572 kB
MemFree: 1286376 kB
MemShared: 0 kB
Buffers: 20216 kB
Cached: 189996 kB
SwapCached: 0 kB
Active: 56644 kB
Inactive: 153648 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 2025572 kB
LowFree: 1286376 kB
SwapTotal: 0 kB
SwapFree: 0 kB