ElasticSearch is a distributed, RESTful search and analytics engine capable of solving a number of use cases. This connector facilitates automated operations to execute lucene query, get mapping and cluster details.
This document provides information about the ElasticSearch Connector, which facilitates automated interactions, with a ElasticSearch server using FortiSOAR™ playbooks. Add the ElasticSearch Connector as a step in FortiSOAR™ playbooks and perform automated operations with ElasticSearch.
Connector Version: 3.0.0
FortiSOAR™ Version Tested on: 7.5.0-4015
ElasticSearch Version Tested on: 8.12.2
Authored By: Fortinet
Certified: Yes
Following enhancements have been made to the ElasticSearch Connector in version 3.0.0:
use_ssl was always true irrespective of the protocol used. Now use_ssl is considered only in the case of HTTPS protocol.ignore_unavailable parameter in code so the specified concrete indices are ignored when unavailable (missing or closed).NOTE: The parameter Type in actions Get Mapping and Execute Query is going to be deprecated in the next connector version.
Use the Content Hub to install the connector. For the detailed procedure to install a connector, click here.
You can also use the yum command as a root user to install the connector:
yum install cyops-connector-elasticsearch
For the procedure to configure a connector, click here
In FortiSOAR™, on the Connectors page, click the ElasticSearch connector row (if you are in the Grid view on the Connectors page) and in the Configurations tab enter the required configuration details:
| Parameter | Description |
|---|---|
| Server URL | The FQDN or IP address of the ElasticSearch server to which you will connect and perform the automated operations. |
| Protocol | The Protocol used to connect to the ElasticSearch server remotely. Choose between HTTP or HTTPS. By default, HTTPS is used. |
| Username | The Username to access the ElasticSearch server to which you will connect and perform the automated operations. |
| Password | The Password to access the ElasticSearch server to which you will connect and perform the automated operations. |
| Port | The Port number that is used to connect to the ElasticSearch server By default, this is set to 9200. |
| Maximum Retries | The maximum number of retries that the connector will make to connect to the ElasticSearch server in the case of network issues. By default, the connection timeout default value is set as 30 seconds. |
| Verify SSL | Specifies whether the SSL certificate for the server is to be verified. By default, this option is set to True. |
The following automated operations can be included in playbooks and you can also use the annotations to access operations:
| Function | Description | Annotation and Category |
|---|---|---|
| Get Cluster Details | Retrieves cluster details and the health information about the cluster configured on your ElasticSearch instance, based on the index you have specified. | get_cluster_details Investigation |
| Get Mapping | Retrieves a list of indices, which includes the type of indices that are currently configured on your ElasticSearch instance. | get_mapping Investigation |
| Execute Query | Executes a query and fetches data from your ElasticSearch instance, based on the input filters. | execute_query Investigation |
| Execute Lucene Query | Executes a search query without using Elastic's DSL Query Syntax. An example query to search for the term phishing in documents where the status field is set to 200 would look like this: phishing AND status:200 |
execute_lucene_query Investigation |
| Get Saved Search | Retrieves details of saved searches from ElasticSearch or Kibana. | get_saved_search Investigation |
| Parameter | Description |
|---|---|
| Index | Specify a particular index whose cluster details you want to retrieve from ElasticSearch. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. |
| Run As User | Specify the user being impersonated for retrieving cluster details. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"cluster_details": {
"routing_nodes": {},
"metadata": {},
"cluster_name": "",
"version": "",
"master_node": "",
"routing_table": {},
"state_uuid": "",
"nodes": {},
"blocks": {},
"snapshots": {}
}
}
| Parameter | Description |
|---|---|
| Index | Specify a particular index whose indices you want to retrieve from ElasticSearch. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. To search for all indices, which are currently configured on your ElasticSearch instance, use _all. |
| Type | Specify the type of indices for which you want to retrieve details. A Type in ElasticSearch represents a class of similar documents. |
| Run As User | Specify the user being impersonated for retrieving indices. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"mapping": {}
}
| Parameter | Description |
|---|---|
| Query | Specify a stringified JSON formatted query used for searching data in ElasticSearch. For example: {"id": "template_1", "params": {"query_string": "search for these words" }} |
| Index | Specify a particular index, based on which you want to run the query. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. |
| Type | Specify the type of indices, based on which you want to run the query. Type in ElasticSearch represents a class of similar documents. |
| Routing | Specify the name of the shard for which you want to retrieve data. |
| Run As User | Specify the user being impersonated for executing the query. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"_shards": "",
"took": "",
"hits": "",
"aggregations": "",
"timed_out": ""
}
| Parameter | Description |
|---|---|
| Query | Specify a single-line Lucene query used for searching data in ElasticSearch. For example: EventID: "1" AND ParentImage: "C:\Program Files\*" |
| Index | Specify a particular index based on which you want to run the query. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. |
| Run As User | Specify the user being impersonated for executing the query. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"_shards": "",
"took": "",
"hits": "",
"aggregations": "",
"timed_out": ""
}
| Parameter | Description |
|---|---|
| Object ID | Specify the ID of the object whose saved search details you want to retrieve from ElasticSearch or Kibana. |
| Port | Specify the port of the Kibana server from which you want to retrieve details of the saved searches. |
The output contains the following populated JSON schema:
{
"version": "",
"id": "",
"updated_at": "",
"type": "",
"attributes": {}
}
The Sample - ElasticSearch - 3.0.0 playbook collection comes bundled with the ElasticSearch connector. These playbooks contain steps using which you can perform all supported actions. You can see bundled playbooks in the Automation > Playbooks section in FortiSOAR™ after importing the ElasticSearch connector.
Note: If you are planning to use any of the sample playbooks in your environment, ensure that you clone those playbooks and move them to a different collection since the sample playbook collection gets deleted during connector upgrade and delete.
ElasticSearch is a distributed, RESTful search and analytics engine capable of solving a number of use cases. This connector facilitates automated operations to execute lucene query, get mapping and cluster details.
This document provides information about the ElasticSearch Connector, which facilitates automated interactions, with a ElasticSearch server using FortiSOAR™ playbooks. Add the ElasticSearch Connector as a step in FortiSOAR™ playbooks and perform automated operations with ElasticSearch.
Connector Version: 3.0.0
FortiSOAR™ Version Tested on: 7.5.0-4015
ElasticSearch Version Tested on: 8.12.2
Authored By: Fortinet
Certified: Yes
Following enhancements have been made to the ElasticSearch Connector in version 3.0.0:
use_ssl was always true irrespective of the protocol used. Now use_ssl is considered only in the case of HTTPS protocol.ignore_unavailable parameter in code so the specified concrete indices are ignored when unavailable (missing or closed).NOTE: The parameter Type in actions Get Mapping and Execute Query is going to be deprecated in the next connector version.
Use the Content Hub to install the connector. For the detailed procedure to install a connector, click here.
You can also use the yum command as a root user to install the connector:
yum install cyops-connector-elasticsearch
For the procedure to configure a connector, click here
In FortiSOAR™, on the Connectors page, click the ElasticSearch connector row (if you are in the Grid view on the Connectors page) and in the Configurations tab enter the required configuration details:
| Parameter | Description |
|---|---|
| Server URL | The FQDN or IP address of the ElasticSearch server to which you will connect and perform the automated operations. |
| Protocol | The Protocol used to connect to the ElasticSearch server remotely. Choose between HTTP or HTTPS. By default, HTTPS is used. |
| Username | The Username to access the ElasticSearch server to which you will connect and perform the automated operations. |
| Password | The Password to access the ElasticSearch server to which you will connect and perform the automated operations. |
| Port | The Port number that is used to connect to the ElasticSearch server By default, this is set to 9200. |
| Maximum Retries | The maximum number of retries that the connector will make to connect to the ElasticSearch server in the case of network issues. By default, the connection timeout default value is set as 30 seconds. |
| Verify SSL | Specifies whether the SSL certificate for the server is to be verified. By default, this option is set to True. |
The following automated operations can be included in playbooks and you can also use the annotations to access operations:
| Function | Description | Annotation and Category |
|---|---|---|
| Get Cluster Details | Retrieves cluster details and the health information about the cluster configured on your ElasticSearch instance, based on the index you have specified. | get_cluster_details Investigation |
| Get Mapping | Retrieves a list of indices, which includes the type of indices that are currently configured on your ElasticSearch instance. | get_mapping Investigation |
| Execute Query | Executes a query and fetches data from your ElasticSearch instance, based on the input filters. | execute_query Investigation |
| Execute Lucene Query | Executes a search query without using Elastic's DSL Query Syntax. An example query to search for the term phishing in documents where the status field is set to 200 would look like this: phishing AND status:200 |
execute_lucene_query Investigation |
| Get Saved Search | Retrieves details of saved searches from ElasticSearch or Kibana. | get_saved_search Investigation |
| Parameter | Description |
|---|---|
| Index | Specify a particular index whose cluster details you want to retrieve from ElasticSearch. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. |
| Run As User | Specify the user being impersonated for retrieving cluster details. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"cluster_details": {
"routing_nodes": {},
"metadata": {},
"cluster_name": "",
"version": "",
"master_node": "",
"routing_table": {},
"state_uuid": "",
"nodes": {},
"blocks": {},
"snapshots": {}
}
}
| Parameter | Description |
|---|---|
| Index | Specify a particular index whose indices you want to retrieve from ElasticSearch. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. To search for all indices, which are currently configured on your ElasticSearch instance, use _all. |
| Type | Specify the type of indices for which you want to retrieve details. A Type in ElasticSearch represents a class of similar documents. |
| Run As User | Specify the user being impersonated for retrieving indices. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"mapping": {}
}
| Parameter | Description |
|---|---|
| Query | Specify a stringified JSON formatted query used for searching data in ElasticSearch. For example: {"id": "template_1", "params": {"query_string": "search for these words" }} |
| Index | Specify a particular index, based on which you want to run the query. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. |
| Type | Specify the type of indices, based on which you want to run the query. Type in ElasticSearch represents a class of similar documents. |
| Routing | Specify the name of the shard for which you want to retrieve data. |
| Run As User | Specify the user being impersonated for executing the query. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"_shards": "",
"took": "",
"hits": "",
"aggregations": "",
"timed_out": ""
}
| Parameter | Description |
|---|---|
| Query | Specify a single-line Lucene query used for searching data in ElasticSearch. For example: EventID: "1" AND ParentImage: "C:\Program Files\*" |
| Index | Specify a particular index based on which you want to run the query. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards. |
| Run As User | Specify the user being impersonated for executing the query. Run As User is a kind of delegation of an account, i.e., the account access is delegated to a different account or an admin account. For example, User X takes on the role of User Y with the help of the run as user mechanism. |
The output contains the following populated JSON schema:
{
"_shards": "",
"took": "",
"hits": "",
"aggregations": "",
"timed_out": ""
}
| Parameter | Description |
|---|---|
| Object ID | Specify the ID of the object whose saved search details you want to retrieve from ElasticSearch or Kibana. |
| Port | Specify the port of the Kibana server from which you want to retrieve details of the saved searches. |
The output contains the following populated JSON schema:
{
"version": "",
"id": "",
"updated_at": "",
"type": "",
"attributes": {}
}
The Sample - ElasticSearch - 3.0.0 playbook collection comes bundled with the ElasticSearch connector. These playbooks contain steps using which you can perform all supported actions. You can see bundled playbooks in the Automation > Playbooks section in FortiSOAR™ after importing the ElasticSearch connector.
Note: If you are planning to use any of the sample playbooks in your environment, ensure that you clone those playbooks and move them to a different collection since the sample playbook collection gets deleted during connector upgrade and delete.