This article describes the features and functionality of CounterMeasures. CounterMeasures are automated remediation and mitigation actions triggered by the FortiMonitor Agent whenever a metric threshold is crossed for more than a configured amount of time. Actions range from simple debugging commands (top,netstat, or vmstat) to complex actions such as making API calls or restarting a service when a threshold is breached. CounterMeasures can increase the level of automation within your operations workflow, either by using the out of the box FortiMonitor CounterMeasures or by creating your own.

CounterMeasures are represented throughout the control panel by a beaker icon. This icon indicating a configured CounterMeasure is available anywhere an incident is listed. You can also see a list of all the configured CounterMeasures in your account by going to Settings > CounterMeasures.

Slack integration

One of the biggest benefits of CounterMeasures is the ability to quickly arm yourself with information when an incident arises. You can configure CounterMeasures to send the information it retrieves directly in Slack, saving you the time to log into the offending instance. See CounterMeasures Slack Integration for more information.

Enable CounterMeasures

Go to the following sections to learn more about enabling CounterMeasures on your instances:

• Linux CounterMeasures

• Windows CounterMeasures

Example workflow

CounterMeasures have a simple workflow that is driven by the FortiMonitor Agent. Here is an example:

1. Enable CounterMeasures in your Agent configuration file or manifest file.

2. Add a CounterMeasure to a metric threshold, such as Disk % used.

3. If the threshold is crossed, the CounterMeasure will be run.

Any output provided by the CounterMeasure will be attached to the incident record and will be available in the FortiMonitor control panel, incident log, or Slack.

CounterMeasure actions

Standard Linux CounterMeasure actions

Out of the box, the FortiMonitor Agent comes with standard CounterMeasure actions that can be utilized.

Standard Windows CounterMeasure actions

The following CounterMeasures are available if you have the Windows Agent installed.

Custom actions

In addition to the standard CounterMeasure actions, you can also create your own by following the steps detailed in:

• Custom Linux CounterMeasure actions

• Custom Windows CounterMeasures actions

Remote actions

You can centrally manage your CounterMeasure plugins by adding the countermeasures_remote_plugins parameter to your Agent manifest file that points to an archive of Agent plugins. When the Agent is installed, it will download and extract the directory in the proper location. Zip, tar, and python file types are supported.

If you would like updates to your CounterMeasures plugins to automatically be applied, you can optionally set countermeasures_refresh_plugins to the number of hours for the refresh interval. This will help with keeping all your servers up-to-date.

See Utilizing Remote Actions for more information.

CounterMeasure metric threshold

CounterMeasures are configured together with metric thresholds. See Configure a CounterMeasure to configure CounterMeasures and thresholds. An orange beaker icon indicates that an approval is required. Selecting the icon displays a modal where you can approve the appropriate CounterMeasure.

Approval

An Approval option is available for situations where you don't want a CounterMeasure to execute without a final authorization. To require approval for CounterMeasure actions, select the Require Approval option when configuring the CounterMeasure.

CounterMeasure status

The CounterMeasureslife cycle is detailed in the following list:

• Pending: state of an action that is scheduled to execute in the future and does not require approval or has already been approved. The beaker icon will appear grey.

• Pending Approval: state of an action that is scheduled to execute in the future but will not do so until approved. The beaker icon will glow orange.

• In Process: state of an action that has been synced down to the Agent to execute. The beaker icon will appear gray.

• Complete: state of an action that has executed successfully. The beaker icon will appear green.

• Error: state of an action that attempted to execute but returned an error while executing. The beaker icon will appear red.

• Skipped: state of an action that was at one time pending but whose underlying incident was resolved before executing. The beaker icon will appear grey.

CounterMeasure output

You can view a CounterMeasure's output from the following:

• CounterMeasures details panel

• Incident details page

The following figure shows the CounterMeasure output from the outage log.

See Viewing CounterMeasure output for more information.

Billing

Your account is only charged for CounterMeasures when they are actively configured on an instance. That is, only instances that have CounterMeasures configured on Alert Thresholds.