Elasticsearch v4.0.0

About the connector

ElasticSearch is a distributed, RESTful search and analytics engine capable of solving a number of use cases. This connector facilitates automated operations to execute lucene query, get mapping and cluster details.

This document provides information about the ElasticSearch Connector, which facilitates automated interactions, with a ElasticSearch server using FortiSOAR™ playbooks. Add the ElasticSearch Connector as a step in FortiSOAR™ playbooks and perform automated operations with ElasticSearch.

Version information

Connector Version: 4.0.0

FortiSOAR™ Version Tested on: 7.6.1-5275

ElasticSearch Version Tested on: 8.17.4

Authored By: Fortinet

Certified: Yes

Release Notes for version 4.0.0

Following enhancements have been made to the ElasticSearch Connector in version 4.0.0:

• Added the following new actions:
  • Get Alerts List
  • Set Alert Status
  • Apply Alert Tags
  • Assign or Unassign Users from Alerts
• Added support for data ingestion to fetch alerts from ElasticSearch/Kibana.

Installing the connector

Use the Content Hub to install the connector. For the detailed procedure to install a connector, click here.

You can also use the yum command as a root user to install the connector:

yum install cyops-connector-elasticsearch

Prerequisites to configuring the connector

• You must have the Server URL and credentials of ElasticSearch server to which you will connect and perform automated operations.
• The FortiSOAR™ server should have outbound connectivity to port 443 on the ElasticSearch server.

Minimum Permissions Required

• Not applicable

Configuring the connector

For the procedure to configure a connector, click here

Configuration parameters

In FortiSOAR™, on the Connectors page, click the ElasticSearch connector row (if you are in the Grid view on the Connectors page) and in the Configurations tab enter the required configuration details:

Actions supported by the connector

The following automated operations can be included in playbooks and you can also use the annotations to access operations:

operation: Get Cluster Details

Input parameters

Output

The output contains the following populated JSON schema:

{
    "cluster_details": {
        "routing_nodes": {},
        "metadata": {},
        "cluster_name": "",
        "version": "",
        "master_node": "",
        "routing_table": {},
        "state_uuid": "",
        "nodes": {},
        "blocks": {},
        "snapshots": {}
    }
}

operation: Get Mapping

Input parameters

Output

The output contains the following populated JSON schema:

{
    "mapping": {}
}

operation: Execute Query

Input parameters

Output

The output contains the following populated JSON schema:

{
    "_shards": "",
    "took": "",
    "hits": "",
    "aggregations": "",
    "timed_out": ""
}

operation: Execute Lucene Query

Input parameters

Output

The output contains the following populated JSON schema:

{
    "_shards": "",
    "took": "",
    "hits": "",
    "aggregations": "",
    "timed_out": ""
}

operation: Get Saved Search

Input parameters

Output

The output contains the following populated JSON schema:

{
    "version": "",
    "id": "",
    "updated_at": "",
    "type": "",
    "attributes": {}
}

operation: Get Alerts List

Input parameters

Output

The output contains the following populated JSON schema:

{
    "took": "",
    "timed_out": "",
    "_shards": {
        "total": "",
        "successful": "",
        "skipped": "",
        "failed": ""
    },
    "hits": {
        "total": {
            "value": "",
            "relation": ""
        },
        "max_score": "",
        "hits": []
    },
    "aggregations": {
        "oldest": {
            "value": "",
            "value_as_string": ""
        },
        "latest": {
            "value": "",
            "value_as_string": ""
        }
    }
}

operation: Set Alert Status

Input parameters

Output

The output contains the following populated JSON schema:

{
    "took": "",
    "timed_out": "",
    "total": "",
    "updated": "",
    "deleted": "",
    "batches": "",
    "version_conflicts": "",
    "noops": "",
    "retries": {
        "bulk": "",
        "search": ""
    },
    "throttled_millis": "",
    "requests_per_second": "",
    "throttled_until_millis": "",
    "failures": []
}

operation: Apply Alert Tags

Input parameters

Output

The output contains the following populated JSON schema:

{
    "took": "",
    "errors": "",
    "items": [
        {
            "update": {
                "_index": "",
                "_id": "",
                "_version": "",
                "result": "",
                "_shards": {
                    "total": "",
                    "successful": "",
                    "failed": ""
                },
                "_seq_no": "",
                "_primary_term": "",
                "status": ""
            }
        }
    ]
}

operation: Assign or Unassign Users from Alerts

Input parameters

Output

The output contains the following populated JSON schema:

{
    "took": "",
    "timed_out": "",
    "total": "",
    "updated": "",
    "deleted": "",
    "batches": "",
    "version_conflicts": "",
    "noops": "",
    "retries": {
        "bulk": "",
        "search": ""
    },
    "throttled_millis": "",
    "requests_per_second": "",
    "throttled_until_millis": "",
    "failures": []
}

Included playbooks

The Sample - ElasticSearch - 4.0.0 playbook collection comes bundled with the ElasticSearch connector. These playbooks contain steps using which you can perform all supported actions. You can see bundled playbooks in the Automation > Playbooks section in FortiSOAR™ after importing the ElasticSearch connector.

• Execute Lucene Query
• Execute Query
• Get Cluster Details
• Get Mapping
• Get Saved Search
• Get Alerts List
• Set Alert Status
• Apply Alert Tags
• Assign or Unassign Users from Alerts

Note: If you are planning to use any of the sample playbooks in your environment, ensure that you clone those playbooks and move them to a different collection since the sample playbook collection gets deleted during connector upgrade and delete.

Data Ingestion Support

Use the Data Ingestion Wizard to easily ingest data into FortiSOAR™ by pulling alerts from ElasticSearch. Currently, alerts ingested from ElasticSearch is mapped to Alerts in FortiSOAR™. For more information on the Data Ingestion Wizard, see the Connectors Guide in the FortiSOAR™ product documentation.

Configure Data Ingestion

You can configure data ingestion using the Data Ingestion Wizard to seamlessly map the incoming ElasticSearch alerts to FortiSOAR™'s Alerts.

The Data Ingestion Wizard helps you to configure the scheduled pulling of data from ElasticSearch into FortiSOAR™. It also lets you pull some sample data from ElasticSearch using which you can define the mapping of data between ElasticSearch and FortiSOAR™. The mapping of common fields is generally already done by the Data Ingestion Wizard; users are mostly required to only map any custom fields that are added to the alerts.

1. To begin configuring data ingestion, click Configure Data Ingestion on the ElasticSearch connector's Configurations page.

   Click Let's Start by fetching some data, to open the Fetch Data screen.

   

   Sample data is required to create a field mapping between ElasticSearch data and FortiSOAR™. The sample data is pulled from connector actions or ingestion playbooks.

2. On the Fetch Data screen, provide the configurations required to fetch alerts from ElasticSearch.

   Users can filter retrieved alerts by selecting the time since when to retrieve the alerts in the Fetch Alerts Since field, status of the alerts in the Status, and the maximum records to fetch in the Maximum Records to Fetch field. For example, if you choose open as Status and maximum record count as 100, then 100 alerts with the Status open are retrieved from the ElasticSearch/Kibana server.

   The fetched data is used to create a mapping between the alerts from ElasticSearch and FortiSOAR Alerts. Once you have completed specifying the configurations, click Fetch Data.

   

3. On the Field Mapping screen, map the fields of the ingested ElasticSearch alerts to the fields of a Alerts present in FortiSOAR™.

   To map a field, click the key in the sample data to add the Jinja value of the field. For example, to map the uuid parameter of an ingested alert from ElasticSearch to the value parameter of a FortiSOAR™ Alert, click the Value field and then click the uuid field to populate its keys:

   

   For more information on field mapping, see the Data Ingestion chapter in the Connectors Guide in the FortiSOAR™ product documentation. Once you have completed the mapping of fields, click Save Mapping & Continue.

4. (Optional) Use the Scheduling screen to configure schedule-based ingestion, i.e., specify the polling frequency to alerts, so that the content gets pulled from the alerts integration into FortiSOAR™

   On the Scheduling screen, from the Do you want to schedule the ingestion? drop-down list, select Yes.

   In the Configure Schedule Settings section, specify the Cron expression for the schedule. For example, if you want to pull data from ElasticSearch every 5 minutes, click Every X Minute, and in the minute box enter */5. This means that the Threat Intelligence Feeds will be pulled from alerts every 5 minutes.

   

   Once you have completed scheduling, click Save Settings & Continue.

5. The Summary screen displays a summary of the mapping done, and it also contains links to the Ingestion playbooks. Click Done to complete the data ingestion and exit the Data Ingestion Wizard.