Elasticsearch Configuration

FortiSOAR leverages the fast search capability of Elasticsearch for quick text search across all records and files in the FortiSOAR database. FortiSOAR supports externalization of Elasticsearch data. Externalization is indexing of data to an Elasticsearch instance that has the same or higher version of Elasticsearch outside of the FortiSOAR virtual appliance; the steps for which are covered in this chapter.

The minimum version of your Elasticsearch cluster must be 7.0.2, if you want to externalize your ElasticticSearch data.

If you want to externalize your other FortiSOAR PostgreSQL database, see the Externalization of your FortiSOAR PostgreSQL database chapter.

Externalization and Authentication of Elasticsearch

If you require to change the location of your Elasticsearch instance from your local instance to a remote machine, run the following steps on your externalized Elasticseach machine:

1. Use the Elasticsearch documentation to install elasticsearch.
   Important: Ensure that you install the same version of elasticsearch that is currently installed on your FortiSOAR instance on the externalized Elasticseach machine. For example, if you have version elasticsearch v7.2.0 currently on your FortiSOAR instance, then u must install elasticsearch v7.2.0 on the remote machine on which you want to externalize elasticseach.
   
2. Configure elasticsearch to accept connections from outside 'localhost' by updating the 'network.host' and 'discovery.type' variables to network.host: 0.0.0.0' and 'discovery.type: single-node' in the /etc/elasticsearch/elasticsearch.yml file.
3. Run the following commands:
   # mkdir -p /opt/cyops-search
# chmod 755 /opt/cyops-search
4. Copy the /opt/cyops-search/exclude.list file from your FortiSOAR machine to the /opt/cyops-search/exclude.list file on your externalized elasticsearch host.
5. Copy the /etc/elasticsearch/security.policy file from your FortiSOAR machine to the /etc/elasticsearch/security.policy file on your externalized elasticsearch host.
6. Run the following commands:
   # chmod 644 /opt/cyops-search/exclude.list
# chmod 660 /etc/elasticsearch/security.policy
# chown root:elasticsearch /etc/elasticsearch/security.policy
   
7. Append the '-Djava.security.policy=/etc/elasticsearch/security.policy' line to the /etc/elasticsearch/jvm.options file.
8. Start elasticsearch using the following command:
   # systemctl start elasticsearch
9. Open the elasticsearch port in your firewall, using the following command:
   # firewall-cmd --permanent --add-port=9200/tcp
# firewall-cmd --reload
10. (Optional) To enable SSL for elasticsearch, see the https://www.elastic.co/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash blog.
11. Once elasticsearch is externalized, it is recommended that you stop and mask 'localhost' elasticsearch on your FortiSOAR machine, using the following commands:
    # systemctl stop elasticsearch
# systemctl disable elasticsearch
# systemctl mask elasticsearch
    

Once elasticsearch is externalized, update the db_config.yml file, which is located at: /opt/cyops/configs/database/db_config.yml

In the db_config.yml file, you require to update the host and port (if needed) in the elasticsearch section that appears as follows:

elasticsearch: 
  es_host: localhost
  es_port: 9200
  es_user: None
  initial_backoff: 60
  max_backoff: 6000
  secret: None
  ssl_cert_path: ""
  use_ssl: false

To change the location of your Elasticsearch instance from your local instance to a remote machine:

es_host: localhost > Update host value with the hostname or IP address of the remote Elasticsearch machine.

es_port: 9200 > Update the port required to access the remote Elasticsearch machine, if required.

For authentication of Elasticsearch (require X-Pack License):

es_user: None > Update the username that is used to access the remote Elasticsearch machine, if Authentication is enabled on the remote Elasticsearch machine

secret: None > Update the secret (password) that is used to access the remote Elasticsearch machine, if Authentication is enabled on the remote Elasticsearch machine.

You also require to assign ngnix permission to the SSL certificate that you have specified in the db_config.yml file using the following command:

chown nginx:nginx filename.pem

Externalized elasticseach must have SSL enabled for use in the FortiSOAR high availability cluster. Also, ensure that you set the use_ssl flag to 'true' and specify the ssl_cert_path as the path of your external elasticsearch CA certificate.

Migration of Elasticsearch data

Once you complete the externalization of Elasticsearch, you will require to migrate your data from your local instance to the remote Elasticsearch machine.

To migrate the remote Elasticsearch machine run the following command on your FortiSOAR instance as a root user after changing the directory to /opt/cyops-api/:

$ sudo -u nginx php /opt/cyops-api/bin/console app:elastic:create --env="prod"

Troubleshooting

FortiSOAR Search Errors

FortiSOAR Search performs indexing in an asynchronous fashion in the backend. Users could be faced with certain scenarios that could lead to a restart of services, which can cause indexing to stop. In this case, FortiSOAR might display any of the following errors when users are performing a search operation on FortiSOAR:

• Search indexing is in progress. Partial results are returned.
• Search indexing has stopped. You must manually rerun indexing (see product documentation for instructions) or raise a support ticket for the same.
• We are sorry, but the server encountered an error while handling your search request. Please contact your administrator for assistance.

In this case, use the /var/log/cyops/cyops-search/falcon.log log file to check which modules are published and indexed and which modules are yet to be published (pending).

For example, the /var/log/cyops/cyops-search/falcon.log log file will display results as follows:

2019-02-13,11:00:44 INFO blocking_connection: _dispatch_events(): 1445: Module Currently Getting Published: ['attachments']
2019-02-13,11:00:44 INFO blocking_connection: _dispatch_events(): 1445: Indexing for Module: 'attachments' started Total Records to be indexed: '1'
2019-02-13,11:00:49 INFO blocking_connection: _dispatch_events(): 1445: Module: 'attachments' Successful Total Records indexed: '1'
2019-02-13,11:00:49 INFO blocking_connection: _dispatch_events(): 1445: on_publish_message called
2019-02-13,11:00:53 INFO blocking_connection: _dispatch_events(): 1445: creating index with mapping
2019-02-13,11:01:00 INFO blocking_connection: _dispatch_events(): 1445: Module Currently Getting Published: ['emails']
2019-02-13,11:01:02 INFO blocking_connection: _dispatch_events(): 1445: Indexing for Module: 'emails' started Total Records to be indexed: '1'
2019-02-13,11:01:04 INFO blocking_connection: _dispatch_events(): 1445: Module: 'emails' Successful Total Records indexed: '1'

The above example shows the attachments and emails modules currently being indexed and its total number of records. Any failure in indexing any modules will be logged here. You can monitor the progress of this file while the indexing is in progress.

If any module(s) are missing from the published list or if any module has the Publish Module: '<name of module>' Unsuccessful listed in the /var/log/cyops/cyops-search/falcon.log log file; the indicators and tasks modules in our example, then you must manually run the indexing for those module(s) using the following command:

$ sudo -u nginx php bin/console app:elastic:create --env="prod" --index='{"type":["<list of comma-seperated module names that require to be indexed>"]}

For our example, run the following command:

$ sudo -u nginx php bin/console app:elastic:create --env="prod" --index='{"type":["indicators","tasks"]}'