What's New in 7.0.0

• Important Notes

• New Features

• Key Enhancements

• Bug Fixes and Enhancements

• Known Issues

Important Notes

1. For native Elasticsearch and Elastic Cloud deployments, FortiSIEM 7.0.0 supports Elasticsearch versions 7.17 and 8.5. If you are running a lower Elasticsearch version and upgrade to FortiSIEM 7.0.0, then Elasticsearch Queries will not work. Follow these steps to properly upgrade your infrastructure.

   1. Upgrade FortiSIEM to 7.0.0.

   2. Upgrade Elasticsearch version to 7.17 or 8.5.

   3. In Admin > Setup > Storage > Online, redo Test and Deploy.

2. AWS Elasticsearch is not supported since they only support Elasticsearch 7.10, which is lower than the required 7.17.

3. AWS Opensearch is not supported.

4. To support new analytical functions in Elasticsearch, the Painless scripting language is used. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/modules-scripting-painless.html for reference. If you are running Elasticsearch, then add the following line to the Elasticsearch.yml file in every Elasticsearch node and restart the cluster for the changes to take effect. Otherwise, queries will fail.

   script.painless.regex.enabled: true

5. 5.x Collector will not work with FortiSIEM 6.7.2 or later. This step is taken for improved security. Follow these steps to make the 5.x Collectors operational after upgrade.

   1. Upgrade the Supervisor to the latest version: 7.0.0 or higher.

   2. Copy phProvisionCollector.collector from the Supervisor to all 5.x Collectors.

      1. Login to Supervisor.

      2. Run the following command.
         

         scp /opt/phoenix/phscripts/bin/phProvisionCollector.collector root@<Collector_IP>:/opt/phoenix/bin/phProvisionCollector
         

   3. Update 5.x Collector password.

      1. SSH to the Collector.

      2. Run the following command.
         

         phProvisionCollector --update <Organization-user-name> <Organization-user-password> <Supervisor-IP> <Organization-name> <Collector-name>

      3. Make sure the Collector ID and password are present in the file /etc/httpd/accounts/passwds on Supervisors and Workers.

   4. Reboot the Collector.

6. In pre-7.0.0 releases, you can define a Report Design Template for an individual Report, a Report Bundle or a Report Folder under Resources > Reports. In this release, assigning a Report Design Template to a Report and a Report Bundle works correctly, but assigning to a Report Folder does not work. This means:

   • In 7.0.0, you cannot assign a custom Report Design Template to a Report Folder.

   • If you are migrating from an earlier release and you have Custom Design Templates assigned to a Report Folder under Resources > Reports, then the pre-7.0.0 -> 7.0.0 Template migration process will not complete. Note that Template migration happens under the hood, when user logs on to the system for the first time after upgrading to 7.0.0. In this case, you will see the following error message, "Another User is updating all Report Templates on this machine. Please try again later.".

   The following workaround is suggested:

   1. Before upgrading, check if you have any Custom Design Templates assigned to a Report Folder. This can be checked in one of two ways:

      • Login to GUI, go to Resources > Reports, and visit each folder and see if there is a Custom Design Template defined.

      • Alternatively, you can download a zip file with the bash script from here, SSH to the Supervisor as root, copy the script to /tmp and run the command:
        

        /tmp/GetCustomFolderReportDesignTemplate
        

        If you get "Permission denied" error while running the script, then run the following command as root.
        

        chmod 755 /tmp/GetCustomFolderReportDesignTemplate

      • If there no Custom Design Template defined for a Report folder, then the script output will be "No Custom Folder Report Design Template Found. You may proceed to regular upgrade."

      • If there are Custom Design Templates defined for a Report folder, then the script output will be something like:
        

        Found 2 Custom Folder Report Design Templates:
        

        testFolder1
        

        testFolder2
        

   2. Before upgrading, if there are Custom Design Templates defined for a Report folder, then you need to delete them. This can be done in one of two ways:

      • Login to GUI, Go to Resources > Reports, select each folder with Custom Folder Report Design Template, select More > Report Design and click "Revert to Default".

      • Alternatively, you can download a zip file with the bash script from here, SSH to the Supervisor as root, copy the script to /tmp and run the command:

        /tmp/RemoveAllCustomFolderReportDesignTemplate
        

        The script output will be something like:
        

        Deleted 2 Custom Folder Report Design Templates:

        testFolder1

        testFolder2

      You may proceed to regular upgrade. If you have already upgraded, then reload the GUI.

   3. If you have already upgraded without doing the procedures 1 and 2 above, then there are two cases:

      • If you do not have any Custom Design Templates assigned to a Report Folder, then the system will work normally.

      • If you do have Custom Design Templates assigned to one or more Report Folders, then you will see the error when you visit Reports page: "Another User is updating all Report Templates on this machine. Please try again later." In this case, you can download a zip file with the bash script from here, SSH to the Supervisor as root, copy the script to /tmp and run the following command:

        /tmp/RemoveAllCustomFolderReportDesignTemplate

        The script output will be:
        

        Deleted 2 Custom Folder Report Design Templates:
        

        testFolder1
        

        testFolder2

      You may proceed to regular upgrade. If you have already upgraded, then reload the GUI.

      After running the script, simply reload the GUI.
      

7. This release cannot be installed with FIPS option.

8. For Enterprise deployments, while creating a custom Report Bundle, you will see an Error: NumberFormat Exception: For input string: "undefined". You can close this error and proceed to create the report bundle. The error has no impact.

9. The Report Design Templates from pre-7.0.0 releases will be migrated to the new format as required by the new Visual Report Design Editor in 7.0. If your pre-7.0.0 Report Design Template contained a PDF attachment in the middle of the template, then after migration, the PDF attachment will be moved to the end of the PDF document.

10. For Windows and Linux Agents monitoring host performance, CMDB > Monitor Status tab is not populated in GUI.

11. FortiSIEM 7.0.0 and later API documentation is transitioning to https://fndn.fortinet.net/index.php?/fortiapi/2627-fortisiem/. Fortinet recommends checking this link first for the latest API updates.

New Features

This release contains the following new features:

• Visual Report Designer

• New Query Functions

• Machine Learning Workbench

• Incident Investigation Workspace

• Built-in Machine Learning Models

• ClickHouse Event Integrity

• Fortinet Security Fabric Discovery

• FortiEMS Discovery

• FortiEMS Endpoint Tagging

• Windows Agent 5.0.0

• Linux Agent 7.0.0

Visual Report Designer

This release provides a report design editor that shows how the report will look in the PDF document. This is often referred to as a WYSIWYG (What you see is what you get) editor. All the features in the current report designer are available with the following exceptions:

• Sub-section from earlier versions is not supported in 7.0.0

• An image/text can be placed side-by-side with another item (image/chart/text) in 7.0.0

For details on creating and editing reports using the Visual Report Designer, see Designing a Report Template.

When a user logs in for the first time to an upgraded FortiSIEM 7.0.0, existing pre-7.0.0 report and report bundle templates will be automatically converted to the new format. The new format will be identical to the old format except in one case: if a user chose 2 charts for the same report in the pre-7.0.0 template, then the charts will be placed next to each other with a shared legend, in the new 7.0.0 format. If there are 4 or more charts for the same report in the pre-7.0.0 template, then the new 7.0.0 format will display multiple rows of charts, with 2 charts in one row, and a common legend at the end.

New Query Functions

The following enhancements are provided for querying events:

1. Aggregation Functions: COUNT, MEDIAN, MODE, PCTILE, STDDEV, and VARIANCE. These compute specific functions in group-by queries and provide additional insights compared to existing aggregation functions: SUM, AVG, MIN, MAX.

2. Time Window Functions: SMA, EMA. These compute simple and exponential moving averages over a time window.

3. String Manipulation Functions: LEN, TO_UPPER, TO_LOWER, REPLACE, TRIM, LTRIM, RTRIM, SUB_STR, URL_DECODE. These do various string manipulations and may be needed to regularize string valued event attributes.

4. Conversion Functions: TO_INTEGER, TO_DOUBLE, TO_STRING, LOG

5. Extraction Function: EXTRACT. This can extract a value from an event attribute, in case the parser missed this attribute in historical data.

6. Evaluation Function: IF. This function can be used to set a new variable based on whether a logical condition based on event attributes is true or false.

7. Allow functions to be nested up to 5 levels, e.g. COUNT DISTINCT (TO_UPPER(user)))

These functions are only available for ClickHouse and Elasticsearch Queries. See the full description link below for limitations of nesting operations.There is no support for EventDB queries and rules.

For full description of the functions and examples, see Functions in Analytics.

Machine Learning Workbench

This release provides a workflow for users to create machine learning tasks based on the events stored in FortiSIEM. You can run a report to create a dataset for a machine learning task, and then train FortiSIEM to create a machine learning model. Then you schedule an inference job to run periodically, which can detect deviations from the model and create incidents, or send emails. The model can be re-trained periodically or on demand. 4 machine learning tasks are supported: Regression, Classification, Anomaly Detection and Forecasting. Classification will only work if there is a labeled field in the event. For Forecasting jobs, email is sent instead of creating incidents.

Four machine learning jobs can be run locally on the Supervisor/Worker cluster, or on AWS. Each platform supports different machine learning tasks and algorithms. When run locally, machine learning jobs are distributed across the Supervisor/Worker cluster - this means that any of Supervisor or Worker nodes can do the training and inference jobs. In each mode, the user can choose a specific machine learning algorithm or run in Auto mode, where FortiSIEM tries to choose the best algorithm with the optimal parameters. Auto mode takes longer to train as various algorithms and parameter sets are attempted during the optimization process.

For details on how to create a machine learning job, see Machine Learning.

Incident Investigation Workspace

Currently, users investigate an Incident within the List View. It is not easy in this view to correlate this Incident with other related Incidents and the entities involved. In this release, a separate Incident Investigation workspace is provided in Analytics > Investigation. Starting with a root Incident, the user can build a link graph relating that Incident to involved entities (IP, Host, user, process, file) and then recursively to other incidents and related entities. The user can view the timeline of these Incidents and play them in a time ordered fashion to visualize how an attack kill chain is developing. Context is provided for every entity based on information in CMDB, external lookups and events in the FortiSIEM Event Database. It is also possible to run reports, run FortiSOAR playbooks and Connectors to gain further insight into an investigation. Finally, the user can also take a remediation action, create a case locally or in an external ticketing system and clear the Incident. In summary, this Workspace enables the user to stay on this page and fully investigate an Incident and take it to closure.

For details on working with Incident Investigation Workspace, see Investigating Incidents.

Built-in Machine Learning Models

The following specialized Machine Learning models are provided:

1. Login Anomaly Detection via Bipartite Graph Edge Anomaly Algorithm

   This release includes a proprietary Machine Learning algorithm that detects login anomalies by learning the user-to-workstation login patterns and forming dynamic peer user groups with similar login patterns. Users and Workstations are represented using a Bipartite graph. In a Bipartite graph, the sets of nodes can be split into two disjoint sets, in such a way that there are no edges between the nodes within the same set. In this example, Users and Workstations form a Bipartite graph, the edge between a User and a Workstation represents a login, and the edge weight represents the number of logins during a time interval.

   This algorithm is part of the FortiSIEM Machine Learning Workbench, introduced in this release. A system defined Machine Learning job including a login report and the Bipartite Graph Edge Anomaly algorithm, is included in this release. The user needs to train the algorithm using the login data from their environment and then schedule the job to run at periodic intervals to detect anomalies. An Incident triggers when an anomaly is detected, along with a visualization of the anomaly.

   For details on the Bipartite Graph Edge Anomaly algorithm, see Anomaly Detection Algorithms for Local Mode.

   For details on how to train and schedule the Login anomaly detection job, see Running Anomaly Detection Local Mode.

2. Incident Resolution Recommendation

   FortiSIEM provides 2 attributes to record Incident status

   • Incident Resolution: None, True Positive, False Positive

   • Incident Status: Active, System Clear and Manually Cleared

When an Incident triggers, Incident Status is Active and Incident Resolution is None. There are 3 ways an Incident can get resolved:

1. If the Incident turns out to be a false positive, then the user can set Incident Resolution to False Positive and Incident Status to Manually Cleared.

2. The Incident may clear itself because of a clearing condition in the rule. In that case, Incident Resolution is set to True Positive and Incident Status is set to System Cleared.

3. The Incident may be a real issue. In that case, after working through the Issue, the user can set Incident Resolution to True Positive and Incident Status to Manually Cleared.

In this release, FortiSIEM uses a Machine Learning Classification algorithm to learn the Incident Resolution set by the user for Incidents over the last 2 days, and recommends Incident Resolution for new Incidents as they happen. The algorithm runs daily at midnight (12AM) to cover Incidents over the last 2 days. Recommendation is done only for new incidents in real time:

• Incident Resolution is set to True Positive or False Positive.

• A new Incident attribute called Confidence (between 0 and 100) is set, with a higher confidence number implying high confidence on the result.

• Incident Comment is updated with the comment "Resolution set by Machine Learning".

Notes:

1. Only Incident Resolution is set and Incident Status is not modified.

2. This algorithm always runs in the background, and cannot be disabled. It uses a set of Incident attributes as features (including Event Receive Time, Event Type, Reporting Device, Source, Target, Category and MITRE Attack Technique) to make its recommendation.

ClickHouse Event Integrity

This release provides a mechanism to check if event data in ClickHouse has been altered after it is first written to database. This feature is resource intensive and turned off by default. When turned on, checksums are computed per shard and per partition from that day onwards and stored in PostgreSQL database. From Admin > Settings > Database > Event Integrity, the user can check the various checksums and ask FortiSIEM to validate them. If some changes were made to the event data, the on-demand checksum would not match the checksum stored in PostgreSQL database.

For details about configuring and validating ClickHouse Event Integrity, see here.

A tool is provided to calculate checksum for historical data. The tool will compute checksums and store them in PostgreSQL database.

Fortinet Security Fabric Discovery

In earlier releases, FortiSIEM can discover a FortiGate firewall via REST API. The attached FortiSwitches, FortiAPs along with the FortiGate firewall and its configuration are discovered. In this release, this discovery is enhanced to a Security Fabric Discovery, where the following additional items are also discovered:

• Security Risk Rating for the entire Fabric, if the discovered FortiGate firewall is a Fabric root firewall.

• FortiClient User Store for the discovered FortiGate firewall, which is the list of FortiClient devices passing through the firewall.

• Shallow discovery of other FortiGate firewalls in the Fabric. Shallow discovery includes basic information about the firewall and does not include detailed information such as FortiClient User Store, configuration, etc.

In this release, the recommended way to discover the full Security Fabric is to individually discover each FortiGate firewall via REST API. The information from various discoveries is merged and displayed in CMDB.

For details about Security Fabric Discovery, see Fortinet FortiGate Firewall in the External Systems Configuration Guide.

FortiEMS Discovery

In this release, FortiSIEM can discover FortiEMS Servers, managed FortiClient endpoint devices and detailed vulnerabilities for each managed FortiClient endpoint. The vulnerability information is normalized to similar information found by vulnerability scanners.

For details about FortiEMS Discovery, see FortiClient EMS in the External Systems Configuration Guide.

FortiEMS Endpoint Tagging

When an Incident triggers in FortiSIEM and it involves a FortiClient endpoint managed by FortiEMS, then user can associate a tag to the FortiClient endpoint in FortiEMS. A tag can be associated with a rule or manually defined. Tagging/Untagging is done via the remediation framework and can be done Adhoc or automated via the notification policy framework. For automation to work correctly, Fortinet Security fabric Discovery must be performed to associate FortiClient endpoint to the FortiEMS that it is registered to.

For details about FortiEMS endpoint tagging, see the Appendix - FortiEMS Endpoint Tagging.

Windows Agent 5.0.0

1. In previous releases, discovery and performance monitoring for Windows Servers had to be performed via WMI/OMI only, which needed an account to be created on the server for FortiSIEM use. In this release, Windows Agent can perform discovery and performance monitoring, this feature has parity with WMI/OMI based discovery and performance monitoring.
   

   For configuring discovery and performance monitoring for Windows Agent, see Configuring Windows Agent - Monitor settings.

2. DNS Analytical logs are now collected via real time Events Tracing for Windows (ETW) provider. This is done to overcome an issue with the old design where DNS analytical logs can stop when the log size is full, requiring the agent to restart in order to pick up new analytical logs.

Linux Agent 7.0.0

In previous releases, discovery and performance monitoring for Linux Servers had to be performed via SNMP and SSH only, which needed configuration changes on the server for FortiSIEM to setup SNMP and SSH connections. In this release, Linux Agent can perform discovery and performance monitoring, this feature has parity with SNMP and SSH based discovery and performance monitoring.

For configuring discovery and performance monitoring for Linux Agent, see Configuring Linux Agent - Monitor settings.

Key Enhancements

• Enhanced Entity Risk View

• External Threat Intelligence Integration Enhancements

• Elasticsearch 8.5.3 Support

• FortiGate VDOM Based Mitigation

• Rule Enhancements

• GUI Inactivity Timeout Enforcement

• Miscellaneous Enhancements

Enhanced Entity Risk View

The Risk Page is re-designed to provide more context for impacted entity (user or host) along with an activity timeline. See Risk View for more information on the Risk Page.

External Threat Intelligence Integration Enhancements

Two enhancements are included in this release.

1. A python-based framework that can be used to integrate new threat intelligence sources. For details see Python Threat Feed Framework in the Appendix.

2. GUI to show the health of threat intelligence integrations. Information includes Status, Feed, Last Updated, Pulling Schedule, Integration Type, Action and missed data polls because of errors. This enables users to make sure that integrations are running correctly.

Elasticsearch 8.5.3 Support

This release adds support for Elasticsearch 8.5.3.

FortiGate VDOM Based Mitigation

The FortiGate mitigation scripts now work if FortiGate has Virtual Domains (VDOMs) defined. User provides VDOM information and the script uses the VDOM during execution. See step 4 in Creating a Remediation Action for more information.

Rule Enhancements

1. Ability to compare event attributes within the same event, e.g. Source IP = Destination IP or Source IP != Destination IP.

2. Allow expression on the Right hand side of query/rule operator.

GUI Inactivity Timeout Enforcement

GUI inactivity time out is specified in CMDB > User > Idle Timeout. This is correctly enforced in this release. Unless the user is in Dashboard, the user is automatically logged out after the specified timeout if the user does not move the mouse or press a key.

Miscellaneous Enhancements

1. Create a CMDB entry for Cloud Service in CMDB and alert when logs are not being received. Host name is used as the IP Address in the logs. Merge discovered Cloud Services in the CMDB if the IP addresses of the service changes.

2. Show Collector ID in the Org Definition screen.

3. Expand AWS S3 Generic Log ingestion to handle multi-line JSON events (if extension is .json.gz or .json).

4. Support SMTP over SSL on ports 587 and 465.

5. Incident and Case PDF Export content improvements.

6. Added heads up display for CMDB > Users and CMDB > Applications to show the most prevalent users and applications.

7. Create a default CMDB > Users group called "FortiSIEM Users" containing administrative users defined locally in FortiSIEM.

Bug Fixes and Enhancements

Bug ID	Severity	Module	Description
885349	Major	App Server	FortiGuard Malware URL entries with special characters may result in App Server exceptions, which may fill up disk and the Supervisor may stop.
885206	Major	App Server	User may not be able to login to FortiSIEM Manager, due to excessive incident updates from instances.
880937	Major	App Server	When customer has user defined parsers, parser order may change unexpectedly after content update or regular upgrade.
891289	Minor	App Server	In notification email, Identity and Location lookup data is merged across organizations.
879916	Minor	App Server	Unable to view adhoc queries from the Query Status tab when the online storage is Elasticsearch.
877909	Minor	App Server	In CMDB > Device, items cannot be sorted globally.
869411	Minor	App Server	Schedule CMDB Report is blank, if Copy to remote host option is chosen and email setting is not configured.
865069	Minor	App Server	For a user defined via AD Group Role, the manually added Contact information will be deleted after user logs out.
859557	Minor	App Server	Unable to delete user defined Dashboard Slideshow in super/global and orgs.
851691	Minor	App Server	CMDB Report: Sometimes the returned number of rows may depend on the combination of display columns used.
843342	Minor	App Server	Incident Title and name are empty for auto clear incidents triggered by OSPF Neighbor Down Rule.
840694	Minor	App Server	AGENT method disappears from CMDB Discovery Method column when SNMP discovery is re-ran.
803284	Minor	App Server	Customer defined Default email sender in Notification Email gets overwritten after upgrade.
797247	Minor	App Server	A user that logs in via AD Group Role config cannot change the Date Format.
795247	Minor	App Server	A CMDB Device Groups can be deleted if there are devices belonging to this group.
749788	Minor	App Server	Delete/Edit CMDB AD User groups with 100k users fails with 'Undefined' error.
799463	Minor	Data Purger	Detect when Elasticsearch Alias is not created, and then try to create again.
817151	Minor	Disaster Recovery	When removing Disaster Recovery (DR) from cluster, cloud health page is not cleaned up; it contains the old cluster data.
876027	Minor	Discovery	FortiGate discovery API fails due to missing 'status' parameter on one of the API calls.
801608	Minor	Discovery	SNMP SysObjectId cannot be applied when a system defined 'Device Type' is used.
892781	Minor	Event Pulling Agents	Failed to Pull ELB forwarded logs using AWS-S3-WITH-SQS.
862020	Minor	Event Pulling Agents	Generic HTTPS Advanced Poller incorrectly sets lastPollTime window to local time instead of UTC.
788696	Minor	Event Pulling Agents	Azure Compute not working to government cloud; No Azure instance found.
690309	Minor	Event Pulling Agents	Unable to receive logs from Cloud-based Endpoint Solutions such as Bitdefender GravityZone via API.
912165	Minor	GUI	Interface Usage Dashboard: Wrong interface values are mapped when selecting interfaces from second table.
897192	Minor	GUI	When sorting a column in a Resource folder, then going to another Resource folder without that column, a Query Exception will occur.
895959	Minor	GUI	Searching function in Parser XML Editor does not work properly.
885293	Minor	GUI	Users are incorrectly redirected to 'Password reset page' even though password is still valid.
881317	Minor	GUI	Some UEBA tags are not applied.
862834	Minor	GUI	Application Monitoring does not show the correct message when you click on Monitor from CMDB.
860518	Minor	GUI	In Incident List View, switching incidents before trigger event query finishes will show the old incident's triggered events.
847236	Minor	GUI	Kafka Configuration - GUI shows an error when hostname is being saved as a Kafka broker.
845231	Minor	GUI	Elasticsearch Query that uses 'CONTAIN' with value ending with '\' will not complete.
807427	Minor	GUI	Incident HTTP notification test fails due to ':' in protocol string.
806694	Minor	GUI	Collector health page does not update 'collector type' column when the value has changed.
796076	Minor	GUI	In org level, Admin > Device Support > Device Apps -> Group list shows natural ID of custom group instead of Display names.
792520	Minor	GUI	Bar color in CMDB> Devices> Summary> Health Overview does not match with thresholds.
791298	Minor	GUI	VirusTotal connector does not complete when adding 'relationship to include' drop down.
853461	Minor	Linux Agent	Linux Agent fails to start up when IPv6 is disabled on Ubuntu 20.04.5.
905514	Minor	Parser (Data)	FortiGateParser stopped recognizing some FGT messages because of unexpected devid format in log.
893761	Minor	Parser (Data)	WinOSWmiParser parses different 'Process Name' for Security 4624 event.
889725	Minor	Parser (Data)	PaloAltoParser does not parse Source IP, Reason & User for PAN-OS-SYSTEM-generic.
886338	Minor	Parser (Data)	FortiGate parser update because of new devid format.
884941	Minor	Parser (Data)	FortiNAC parser needs to be extended.
877268	Minor	Parser (Data)	Event Type 'Google_Apps_moderator_action_add_user' needs to have more attributes to be parsed.
869873	Minor	Parser (Data)	FortiWeb Event Types contains incorrect description.
865141	Minor	Parser (Data)	Microsoft NPS event not fully parsed.
863302	Minor	Parser (Data)	3 Event Types have severity above 10.
846007	Minor	Parser (Data)	Parsed event type 'SentinelOne-EPP-Generic' missing event attributes.
842119	Minor	Parser (Data)	File Name' attribute incorrect or blank for FortiSandbox Syslog.
840182	Minor	Parser (Data)	WinOSWmiParser does not parse events with id 18456, if there is no user defined at the raw event log.
811131	Minor	Parser (Data)	CiscoIOS Parser has an unknown event.
809815	Minor	Parser (Data)	Palo Alto Threat ID 34261 miscategorized. Should be for cobalt strike, not a benign definition.
798684	Minor	Parser (Data)	Parse Cisco AMP for Endpoints API V0 raw logs for more information.
754074	Minor	Parser (Data)	Update Microsoft Network Policy Manager Parser for Windows Agent Collection.
907902	Minor	Performance Monitoring	Custom Perf Monitors always returns numerical data as DOUBLE, even when it is specified to be of a different data type.
898371	Minor	Performance Monitoring	Fail to monitor WebLogic 12c memory.
871853	Minor	Query	PctChange function is not working.
861224	Minor	RuleWorker	phRuleWorker randomly crashes due to possible memory corruption.
876849	Minor	System	For Disaster Recovery in EventDB based deployments, if NFS takes a long time to respond, replication health page responds incorrectly.
874222	Minor	System	FortiSIEM install fails since Red Hat hypervisor is not explicitly supported in install scripts.
867999	Minor	System	Changing the IP of the Supervisor using configFSM.sh will cause svn_url to change to repos/cmdb/.
857752	Minor	System	Include all cert formats during the Upgrade certificate backup and restore procedures.
729023	Minor	System	SQLite header and source version mismatch causes upgrade failure.
881225	Minor	Windows Agent	Unable to collect Windows DHCP logs with traditional Chinese characters in DhcpSrvLog-Mon.log.
799857	Minor	Windows Agent	XML key is truncated in Windows security events 1202/1203.
856691	Enhancement	Data	For the scenario - Administrator is added to FortiGate, the event type should be properly parsed and a rule should be created.
814287	Enhancement	DataPurger	Enhance Elasticsearch Event Export tool phExportESEvent to include org ID as an argument.
814145	Enhancement	Event Pulling Agents	Support Gzip compressed files on HTTP POST feature.
813609	Enhancement	Event Pulling Agents	Support Tenable Nessus Security Scanner via Nessus10 API.
796857	Enhancement	GUI	Support LookupTableGet() and event attribute on right side of Filter condition.
796453	Enhancement	GUI	Azure EventHub integration missing mapping to organization.
878826	Enhancement	Linux Agent	Add support for Ubuntu 22.04 LTS.
868661	Enhancement	Linux Agent	Add support for CentOS 9, RHEL 9 and Rocky Linux 9.
871607	Enhancement	Parser (Data)	Extend FortiDeceptor parser to include MITRE ATTACK TTP information.
845671	Enhancement	Parser (Data)	Event Severity' is not being parsed and evaluated properly in the KasperskyParser.
811438	Enhancement	Parser (Data)	Add support for cronyd events.
802206	Enhancement	Parser (Data)	Add parser for TSV formatted Zeek log.
845685	Enhancement	System	Unable to update FortiSandbox Malware Hash and URL In STIX v2 format.

Known Issues

• General

• ClickHouse Related

• Discovery Related

• Elasticsearch Related

• EventDB Related

• HDFS Related

• High Availability Related

General

See issues mentioned in Important Notes.

ClickHouse Related

1. If you are running ClickHouse event database and want to do Active-Active Supervisor failover, then your Supervisor should not be the only ClickHouse Keeper node. In that case, once the Supervisor is down, the ClickHouse cluster will be down and inserts will fail. It is recommended that you have 3 ClickHouse Keeper nodes running on Workers.

2. If you are running ClickHouse, then during a Supervisor upgrade to FortiSIEM 6.7.0 or later, instead of shutting down Worker nodes, you need to stop the backend processes by running the following command from the command line.

   phtools --stop all

3. If you are running Elasticsearch or FortiSIEM EventDB and switch to ClickHouse, then you need to follow two steps to complete the database switch.

   1. Set up the disks on each node in ADMIN > Setup> Storage and ADMIN > License > Nodes.

   2. Configure ClickHouse topology in ADMIN > Settings > Database > ClickHouse Config.

4. In a ClickHouse environment, Queries will not return results if none of the query nodes within a shard are reachable from Supervisor and responsive. In other words, if at least 1 query node in every shard is healthy and responds to queries, then query results will be returned. To avoid this condition, make sure all Query Worker nodes are healthy.

Discovery Related

Test Connectivity & Discovery may get stuck with Database update 0% when a few discoveries are running.

Elasticsearch Related

1. In Elasticsearch based deployments, queries containing "IN Group X" are handled using Elastic Terms Query. By default, the maximum number of terms that can be used in a Terms Query is set to 65,536. If a Group contains more than 65,536 entries, the query will fail.

   The workaround is to change the “max_terms_count” setting for each event index. FortiSIEM has been tested up to 1 million entries. The query response time will be proportional to the size of the group.
   

   Case 1. For already existing indices, issue the REST API call to update the setting

   PUT fortisiem-event-*/_settings
   {
     "index" : {
       "max_terms_count" : "1000000"
     }
   }
   

   Case 2. For new indices that are going to be created in the future, update fortisiem-event-template so those new indices will have a higher max_terms_count setting

   1. cd /opt/phoenix/config/elastic/7.7

   2. Add "index.max_terms_count": 1000000 (including quotations) to the “settings” section of the fortisiem-event-template.
      

      Example:

      ...

        "settings": {
          "index.max_terms_count": 1000000,
      

      ...

   3. Navigate to ADMIN > Storage > Online and perform Test and Deploy.

   4. Test new indices have the updated terms limit by executing the following simple REST API call.

      GET fortisiem-event-*/_settings

2. FortiSIEM uses dynamic mapping for Keyword fields to save Cluster state. Elasticsearch needs to encounter some events containing these fields before it can determine their type. For this reason, queries containing group by on any of these fields will fail if Elasticsearch has not seen any event containing these fields. Workaround is to first run a non-group by query with these fields to make sure that these fields have non-null haves.

EventDB Related

Currently, Policy based retention for EventDB does not cover two event categories: (a) System events with phCustId = 0, e.g. a FortiSIEM External Integration Error, FortiSIEM process crash etc., and (b) Super/Global customer audit events with phCustId = 3, e.g. audit log generated from a Super/Global user running an adhoc query. These events are purged when disk usage reaches high watermark.

HDFS Related

If you are running real-time Archive with HDFS, and have added Workers after the real-time Archive has been configured, then you will need to perform a Test and Deploy for HDFS Archive again from the GUI. This will enable HDFSMgr to know about the newly added Workers.

High Availability Related

If you make changes to the following files on any node in the FortiSIEM Cluster, then you will have to manually copy these changes to other nodes.

1. FortiSIEM Config file (/opt/phoenix/config/phoenix_config.txt): If you change a Supervisor (respectively Worker, Collector) related change in this file, then the modified file should be copied to all Supervisors (respectively Workers, Collectors).

2. FortiSIEM Identity and Location Configuration file (/opt/phoenix/config/identity_Def.xml): This file should be identical in Supervisors and Workers. If you make a change to this file on any Supervisor or Worker, then you need to copy this file to all other Supervisors and Workers.

3. FortiSIEM Profile file (ProfileReports.xml): This file should be identical in Supervisors and Workers. If you make a change to this file on any Supervisor or Worker, then you need to copy this file to all other Supervisors and Workers.

4. SSL Certificate (/etc/httpd/conf.d/ssl.conf): This file should be identical in Supervisors and Workers. If you make a change to this file on any Supervisor or Worker, then you need to copy this file to all other Supervisors and Workers.

5. Java SSL Certificates (files cacerts.jks, keyfile and keystore.jks under /opt/glassfish/domains/domain1/config/): If you change these files on a Supervisor, then you have to copy these files to all Supervisors.

6. Log pulling External Certificates: Copy all log pulling external certificates to each Supervisor.

7. Event forwarding Certificates define in FortiSIEM Config file (/opt/phoenix/config/phoenix_config.txt): If you change on one node, you need to change on all nodes.

8. Custom cron job: If you change this file on a Supervisor, then you have to copy this file to all Supervisors.