What's New in 6.7.3

• Important Notes

• Bug Fixes and Enhancements

• Known Issues

Important Notes

5.x Collector will not work with FortiSIEM 6.7.2 or later. This step is taken for improved security. Follow these steps to make the 5.x Collectors operational after upgrade.

1. Upgrade the Supervisor to the latest version: 7.0.0 or higher.

2. Copy phProvisionCollector.collector from the Supervisor to all 5.x Collectors.

   1. Login to Supervisor.

   2. Run the following command.
      

      scp /opt/phoenix/phscripts/bin/phProvisionCollector.collector root@<Collector_IP>:/opt/phoenix/bin/phProvisionCollector
      

3. Update 5.x Collector password.

   1. SSH to the Collector.

   2. Run the following command.

      phProvisionCollector --update <Organization-user-name> <Organization-user-password> <Supervisor-IP> <Organization-name> <Collector-name>
      

   3. Make sure the Collector ID and password are present in the file /etc/httpd/accounts/passwds on Supervisors and Workers.

4. Reboot the Collector.

Bug Fixes and Enhancements

This release contains the following fixes and enhancements.

1. Upgrade of Log4j version from 1.2 to 2.20. Note that FortiSIEM 6.7.2 is not vulnerable to Log4j 1.2 vulnerabilities. This upgrade is a matter of precaution.

2. For installations with a large number of Collectors, App Server resource pools may be used up internally, causing users to not be able to login to the GUI. This is caused by a known issue in the Jackson library for serializing / deserializing JSON objects as noted in https://stackoverflow.com/questions/75616007/at-com-fasterxml-jackson-core-util-interncache-interninterncache-java54-in-bl.

3. Improve the performance of large LDAP discoveries with many users and user groups.

Known Issues

• ClickHouse Related

• Elasticsearch Related

• EventDB Related

• HDFS Related

• High Availability Related

ClickHouse Related

1. If you are running ClickHouse event database and want to do Active-Active Supervisor failover, then your Supervisor should not be the only ClickHouse Keeper node. In that case, once the Supervisor is down, the ClickHouse cluster will be down and inserts will fail. It is recommended that you have 3 ClickHouse Keeper nodes running on Workers.

2. If you are running ClickHouse, then during a Supervisor upgrade to FortiSIEM 6.7.0 or later, instead of shutting down Worker nodes, you need to stop the backend processes by running the following command from the command line.
   

   phtools --stop all
   

3. If you are running Elasticsearch or FortiSIEM EventDB and switch to ClickHouse, then you need to follow two steps to complete the database switch.

   1. Set up the disks on each node in ADMIN > Setup> Storage and ADMIN > License > Nodes.

   2. Configure ClickHouse topology in ADMIN > Settings > Database > ClickHouse Config.

4. In a ClickHouse environment, Queries will not return results if none of the query nodes within a shard are reachable from Supervisor and responsive. In other words, if at least 1 query node in every shard is healthy and responds to queries, then query results will be returned. To avoid this condition, make sure all Query Worker nodes are healthy.

Elasticsearch Related

1. In Elasticsearch based deployments, queries containing "IN Group X" are handled using Elastic Terms Query. By default, the maximum number of terms that can be used in a Terms Query is set to 65,536. If a Group contains more than 65,536 entries, the query will fail.

   The workaround is to change the “max_terms_count” setting for each event index. FortiSIEM has been tested up to 1 million entries. The query response time will be proportional to the size of the group.
   

   Case 1. For already existing indices, issue the REST API call to update the setting

   PUT fortisiem-event-*/_settings
   {
     "index" : {
       "max_terms_count" : "1000000"
     }
   }
   

   Case 2. For new indices that are going to be created in the future, update fortisiem-event-template so those new indices will have a higher max_terms_count setting

   1. cd /opt/phoenix/config/elastic/7.7

   2. Add "index.max_terms_count": 1000000 (including quotations) to the “settings” section of the fortisiem-event-template.
      

      Example:

      ...

        "settings": {
          "index.max_terms_count": 1000000,
      

      ...

   3. Navigate to ADMIN > Storage > Online and perform Test and Deploy.

   4. Test new indices have the updated terms limit by executing the following simple REST API call.

      GET fortisiem-event-*/_settings

2. FortiSIEM uses dynamic mapping for Keyword fields to save Cluster state. Elasticsearch needs to encounter some events containing these fields before it can determine their type. For this reason, queries containing group by on any of these fields will fail if Elasticsearch has not seen any event containing these fields. Workaround is to first run a non-group by query with these fields to make sure that these fields have non-null haves.

EventDB Related

Currently, Policy based retention for EventDB does not cover two event categories: (a) System events with phCustId = 0, e.g. a FortiSIEM External Integration Error, FortiSIEM process crash etc., and (b) Super/Global customer audit events with phCustId = 3, e.g. audit log generated from a Super/Global user running an adhoc query. These events are purged when disk usage reaches high watermark.

HDFS Related

If you are running real-time Archive with HDFS, and have added Workers after the real-time Archive has been configured, then you will need to perform a Test and Deploy for HDFS Archive again from the GUI. This will enable HDFSMgr to know about the newly added Workers.

High Availability Related

If you make changes to the following files on any node in the FortiSIEM Cluster, then you will have to manually copy these changes to other nodes.

1. FortiSIEM Config file (/opt/phoenix/config/phoenix_config.txt): If you change a Supervisor (respectively Worker, Collector) related change in this file, then the modified file should be copied to all Supervisors (respectively Workers, Collectors).

2. FortiSIEM Identity and Location Configuration file (/opt/phoenix/config/identity_Def.xml): This file should be identical in Supervisors and Workers. If you make a change to this file on any Supervisor or Worker, then you need to copy this file to all other Supervisors and Workers.

3. FortiSIEM Profile file (ProfileReports.xml): This file should be identical in Supervisors and Workers. If you make a change to this file on any Supervisor or Worker, then you need to copy this file to all other Supervisors and Workers.

4. SSL Certificate (/etc/httpd/conf.d/ssl.conf): This file should be identical in Supervisors and Workers. If you make a change to this file on any Supervisor or Worker, then you need to copy this file to all other Supervisors and Workers.

5. Java SSL Certificates (files cacerts.jks, keyfile and keystore.jks under /opt/glassfish/domains/domain1/config/): If you change these files on a Supervisor, then you have to copy these files to all Supervisors.

6. Log pulling External Certificates: Copy all log pulling external certificates to each Supervisor.

7. Event forwarding Certificates define in FortiSIEM Config file (/opt/phoenix/config/phoenix_config.txt): If you change on one node, you need to change on all nodes.

8. Custom cron job: If you change this file on a Supervisor, then you have to copy this file to all Supervisors.