Server-policy outage

1. Check if all services on FortiWeb are not available, including the HTTPS/SSH service to the management portal, and the HTTP/HTTPS access to the server-policy;
2. Check the system resources especially when the memory size is low (4G or below, or many configuration entries) or memory usage is abnormal (memory leak or OOM - out of memory);
3. Check if reboot, crash or coredump occurred when the issue happened;
4. Check if any new operation/configuration are performed/added/modified before the issue happened;

   Event logs can be checked for configuration change event, while detailed CLIs are not included.

5. Check if there is any traffic (CPS/Throughput/Attack) burst or shift when the issue happened;

   Traffic burst usually leads to high CPU usage, so you can check the Event logs, nmon records, or 3rd party network monitoring history to confirm.

6. Check if a high volume of logs generated or sent to FortiAnylazer or other outside log servers (may be CPU consuming)

Temporary Actions/Solution

• Check the status of proxyd with ps | grep proxyd;

• Execute exec session-cleanup (recommended), kill proxyd, or kill dnsproxyd (or fn kill proxyd on the front CLI) to restart proxyd or other processes.

• Collect system and debug logs for further support analysis:

  • Most important system logs can be fetched by one-click download via GUI > System > Maintenance > Debug > Download:

    Please note that you need to enable GUI > System > Config > Feature Visibility > Debug before seeing such option:

  • Sometimes newly-added debug logs may not be included in the archive file downloaded through above method, then it’s better to check and download such logs via GUI > System > Maintenance > Backup & Restore > GUI File Download/Upload:

    Similarly, you needs to enable the GUI File Download/Upload via CLI:

    config system settings

    set enable-file-upload enable

    end