Fortinet black logo

What's New in 7.1.6

What's New in 7.1.6

This document describes the content on the FortiSIEM 7.1.6 release.

OS Updates

Rocky Linux Update

This release includes published Rocky Linux OS 8.9 updates until May 6, 2024. The list of updates can be found at https://errata.rockylinux.org/. FortiSIEM Rocky Linux Repositories (os-pkgs-cdn.fortisiem.fortinet.com and os-pkgs-r8.fortisiem.fortinet.com) have also been updated to include fixes until May 6, 2024. FortiSIEM customers in versions 6.4.1 and above, can upgrade only their Rocky Linux versions by following the procedures described in FortiSIEM OS Update Procedure.

PostGreSQL Update

FortiSIEM 7.1.6 includes PostGreSQL v13.14 containing the patch for CVE-2024-0985.

  • If you are doing a fresh install of FortiSIEM 7.1.6, then the patch is included and there is nothing to do.

  • If are upgrading to FortiSIEM 7.1.6, then the patch is included and there is nothing to do.

  • If you want to remain on a version of FortiSIEM 7.1.4 or earlier, then you can't get this patch by running yum upgrade, since Postgres changed the repo gpg key as per this change
    (https://yum.postgresql.org/news/pgdg-rpm-repo-gpg-key-update/). To get this Postgres patch, on the Supervisor, run the following script:

curl -s https://os-pkgs-cdn.fortisiem.fortinet.com/postgres/misc/switch-pgdg-repo-and-upgrade-to-pg13.14.sh | bash -xe

Bug Fixes

This release contains the following fixes.

Bug Id

Severity

Module

Description

1016660, 1006507

Major

App Server

Supervisor will stop uploading incidents to FortiSIEM Manager if there are too many incidents backed up on Supervisor.

1023244

Major

Disaster Recovery

For ClickHouse in Disaster Recovery Environment, after switching to Secondary, historical queries may fail.

1010268

Major

Generative AI

Fortinet Advisor response shows information from other organizations.

1022966

Major

Parser

Improve phParser Syslog over TLS server processing to handle more clients.

1013804

Major

Query

EventDB deployments: Queries with any group by large field length value (> 4K) fails with 'Search Data Error'.

1025920

Major

Rule

Improve RuleWorker to reduce event drops during heavy load. Event drops produce PH_DROP_EVENT_FROM_SHARED_BUFFER error events.

1002055

Major

Rule

Sometimes for Service provider Organizations, disabled rules may trigger incidents after upgrade to 7.1.4.

938498

Major

Rule

Sometimes phRuleWorker crashes on workers.

1003855

Major

Rule, Report

phQueryWorker, phRuleWorker and phReportWorker processes may crash when group by attribute is too long.

1026841

Major

System

Collector may fail to upload config events due to mod_security rules.

1025670

Minor

App Server

Raw Events in email incident notification (via custom template) is truncated when Incident contains Windows Agent events.

1022684

Minor

App Server

Optimize App Server handling of incident detail and lookup-time updates in System config for Elasticsearch deployments.

1022485

Minor

App Server

Windows Agents fail authentication with Supervisor in VDI environment.

1021956

Minor

App Server

Remove Organizations from other CMDB tables (ph_drq_scope) after an Organization is deleted from CMDB. This error can cause disabled rules to get activated if the rule was active in the deleted organization.

1020274

Minor

App Server

Investigation View doesn't load some incidents if there are 2 devices in Incident with the same name.

1019109

Minor

App Server

Improve App Server Task retrieval performance from Redis. This can cause GUI to become slow with large number of collectors.

1015817

Minor

App Server

IPS CVE Check Integration is broken - Cannot get CVE list from FortiGuard Services.

1012404

Minor

App Server

Org level admin cannot create System Rule Exception from Org scope.

997457

Minor

App Server

After decommissioning an unmanaged device, create a new managed device with same IP and its logs are dropped.

995640

Minor

App Server

Rule import from XML can bypass the rule name restriction that a rule name can't start with numbers.

995638

Minor

App Server

CSV Export of Watchlist does not handle comma's inside value (even if double quoted).

987938

Minor

App Server

phRecvDate cannot be used in GroupByAttr when using rest/query/eventQuery.

978060

Minor

App Server

Notification policies with Remediation script is configured to run on the collector, but is running on Supervisor.

905928

Minor

App Server

Attachments are not received in Case email notification.

1026042

Minor

App Server, Upgrade

During Collector Upgrade, DownloadImage task for a particular collector status may stay in Inwaiting stage and may not proceed.

1025801

Minor

ClickHouse Backend

ClickHouse Log Integrity Validation does not work from Supervisor Follower.

990441

Minor

ClickHouse Backend

ClickHouse Server does not clean up old unused AWS S3 objects occupying storage space.

987286

Minor

ClickHouse Backend

ClickHouse - Add worker > Test may fail if the disk is already formatted.

1022689

Minor

Docker, System

Linux agent event can't be uploaded through Docker Collector.

1024358

Minor

GUI

Duplicate 'Incident First Seen' show under CMDB Report Attribute list.

1023547

Minor

GUI

Some Lookup Tables can not be imported in Enterprise version.

1021133

Minor

GUI

Severity of more than 1 selected system rules cannot be changed WITHOUT creating a clone rule.

1011914

Minor

GUI

Incident > 'Run External Integration' Action is missing.

1007538

Minor

GUI

During rule editing, removing an attribute from GroupBy in Filter Conditions (Step 2) does not remove this attribute from Action (Step 3) - this causes rulesync issues.

1005651

Minor

GUI

Incident Explorer View is missing 'Other' Category.

997725

Minor

GUI

User GUI timeout in background browser tab logs out active foreground tab.

983266

Minor

GUI

GUI removes spaces between words in 'Destination Organization' and likely other attributes.

956930

Minor

GUI

After deleting Organizations and creating new one, user sometimes fails to create archive policies for new organizations.

991802

Minor

Linux Agent

Linux agent on Ubuntu 22.04.3 does not start when using Supervisor FQDN. Using Supervisor IP works.

987004

Minor

Linux Agent, Parser

Linux agent 7.1.0 fails to parse Process names with spaces.

1006302

Minor

Machine Learning

The Gaussian Model and Gaussian Mixture Models algorithms do not handle large data properly because of missing data normalization.

876167

Minor

Parser

During Testing Rules, parsed event attribute values containing semicolon do not show correctly.

936491

Minor

Performance Monitoring

After cloning a JDBC Performance Object, Test may fail.

1021803, 1004011

Minor

Query

For ClickHouse, there may be result discrepancy between Search on GUI and CSV Export.

996566

Minor

Query

For ClickHouse, it is not possible to do a group by on event attributes with Uint64 values.

1025395

Minor

System

Restarting 'phMonitor' would result in the unnecessarily restarting 'ClickHouseServer'.

1018855

Minor

System

For phClickHouseCSVExport tool, device ip argument should mean reporting ip.

938732

Minor

Upgrade

When collector upgrade fails, phMonitor.pre-upgrade should move back to phMonitor.

1026025

Enhancement

Data work

FortiMail Parser does not parse logs from FortiCloud FortiMail.

1024781

Enhancement

Data work

Win-Security-4688 is not parsing CommandLine in new WinOSXmlParser.

1007504

Enhancement

GUI

Interface Usage Dashboard data cannot be sorted.

996423

Enhancement

GUI

Show Organization names in alphabetical order.

989168

Enhancement

System

Support change_ip operation for worker and non-licensed Supervisor.

Post-Upgrade ClickHouse IP Index Rebuilding

If you are upgrading ClickHouse based deployment from pre-7.1.1 to 7.1.6, then after upgrading to 7.1.6, you need to run a script to rebuild ClickHouse indices. If you are running 7.1.2, 7.1.3, 7.1.4, or 7.1.5, and have already executed the rebuilding steps, then nothing more needs to be done.

For details about this issue, see Release Notes 7.1.3 Known Issue.

The rebuilding steps are available in Release Notes 7.1.4 - Script for Rebuilding/Recreating pre-7.1.1 ClickHouse Database Indices Involving IP Fields.

What's New in 7.1.6

This document describes the content on the FortiSIEM 7.1.6 release.

OS Updates

Rocky Linux Update

This release includes published Rocky Linux OS 8.9 updates until May 6, 2024. The list of updates can be found at https://errata.rockylinux.org/. FortiSIEM Rocky Linux Repositories (os-pkgs-cdn.fortisiem.fortinet.com and os-pkgs-r8.fortisiem.fortinet.com) have also been updated to include fixes until May 6, 2024. FortiSIEM customers in versions 6.4.1 and above, can upgrade only their Rocky Linux versions by following the procedures described in FortiSIEM OS Update Procedure.

PostGreSQL Update

FortiSIEM 7.1.6 includes PostGreSQL v13.14 containing the patch for CVE-2024-0985.

  • If you are doing a fresh install of FortiSIEM 7.1.6, then the patch is included and there is nothing to do.

  • If are upgrading to FortiSIEM 7.1.6, then the patch is included and there is nothing to do.

  • If you want to remain on a version of FortiSIEM 7.1.4 or earlier, then you can't get this patch by running yum upgrade, since Postgres changed the repo gpg key as per this change
    (https://yum.postgresql.org/news/pgdg-rpm-repo-gpg-key-update/). To get this Postgres patch, on the Supervisor, run the following script:

curl -s https://os-pkgs-cdn.fortisiem.fortinet.com/postgres/misc/switch-pgdg-repo-and-upgrade-to-pg13.14.sh | bash -xe

Bug Fixes

This release contains the following fixes.

Bug Id

Severity

Module

Description

1016660, 1006507

Major

App Server

Supervisor will stop uploading incidents to FortiSIEM Manager if there are too many incidents backed up on Supervisor.

1023244

Major

Disaster Recovery

For ClickHouse in Disaster Recovery Environment, after switching to Secondary, historical queries may fail.

1010268

Major

Generative AI

Fortinet Advisor response shows information from other organizations.

1022966

Major

Parser

Improve phParser Syslog over TLS server processing to handle more clients.

1013804

Major

Query

EventDB deployments: Queries with any group by large field length value (> 4K) fails with 'Search Data Error'.

1025920

Major

Rule

Improve RuleWorker to reduce event drops during heavy load. Event drops produce PH_DROP_EVENT_FROM_SHARED_BUFFER error events.

1002055

Major

Rule

Sometimes for Service provider Organizations, disabled rules may trigger incidents after upgrade to 7.1.4.

938498

Major

Rule

Sometimes phRuleWorker crashes on workers.

1003855

Major

Rule, Report

phQueryWorker, phRuleWorker and phReportWorker processes may crash when group by attribute is too long.

1026841

Major

System

Collector may fail to upload config events due to mod_security rules.

1025670

Minor

App Server

Raw Events in email incident notification (via custom template) is truncated when Incident contains Windows Agent events.

1022684

Minor

App Server

Optimize App Server handling of incident detail and lookup-time updates in System config for Elasticsearch deployments.

1022485

Minor

App Server

Windows Agents fail authentication with Supervisor in VDI environment.

1021956

Minor

App Server

Remove Organizations from other CMDB tables (ph_drq_scope) after an Organization is deleted from CMDB. This error can cause disabled rules to get activated if the rule was active in the deleted organization.

1020274

Minor

App Server

Investigation View doesn't load some incidents if there are 2 devices in Incident with the same name.

1019109

Minor

App Server

Improve App Server Task retrieval performance from Redis. This can cause GUI to become slow with large number of collectors.

1015817

Minor

App Server

IPS CVE Check Integration is broken - Cannot get CVE list from FortiGuard Services.

1012404

Minor

App Server

Org level admin cannot create System Rule Exception from Org scope.

997457

Minor

App Server

After decommissioning an unmanaged device, create a new managed device with same IP and its logs are dropped.

995640

Minor

App Server

Rule import from XML can bypass the rule name restriction that a rule name can't start with numbers.

995638

Minor

App Server

CSV Export of Watchlist does not handle comma's inside value (even if double quoted).

987938

Minor

App Server

phRecvDate cannot be used in GroupByAttr when using rest/query/eventQuery.

978060

Minor

App Server

Notification policies with Remediation script is configured to run on the collector, but is running on Supervisor.

905928

Minor

App Server

Attachments are not received in Case email notification.

1026042

Minor

App Server, Upgrade

During Collector Upgrade, DownloadImage task for a particular collector status may stay in Inwaiting stage and may not proceed.

1025801

Minor

ClickHouse Backend

ClickHouse Log Integrity Validation does not work from Supervisor Follower.

990441

Minor

ClickHouse Backend

ClickHouse Server does not clean up old unused AWS S3 objects occupying storage space.

987286

Minor

ClickHouse Backend

ClickHouse - Add worker > Test may fail if the disk is already formatted.

1022689

Minor

Docker, System

Linux agent event can't be uploaded through Docker Collector.

1024358

Minor

GUI

Duplicate 'Incident First Seen' show under CMDB Report Attribute list.

1023547

Minor

GUI

Some Lookup Tables can not be imported in Enterprise version.

1021133

Minor

GUI

Severity of more than 1 selected system rules cannot be changed WITHOUT creating a clone rule.

1011914

Minor

GUI

Incident > 'Run External Integration' Action is missing.

1007538

Minor

GUI

During rule editing, removing an attribute from GroupBy in Filter Conditions (Step 2) does not remove this attribute from Action (Step 3) - this causes rulesync issues.

1005651

Minor

GUI

Incident Explorer View is missing 'Other' Category.

997725

Minor

GUI

User GUI timeout in background browser tab logs out active foreground tab.

983266

Minor

GUI

GUI removes spaces between words in 'Destination Organization' and likely other attributes.

956930

Minor

GUI

After deleting Organizations and creating new one, user sometimes fails to create archive policies for new organizations.

991802

Minor

Linux Agent

Linux agent on Ubuntu 22.04.3 does not start when using Supervisor FQDN. Using Supervisor IP works.

987004

Minor

Linux Agent, Parser

Linux agent 7.1.0 fails to parse Process names with spaces.

1006302

Minor

Machine Learning

The Gaussian Model and Gaussian Mixture Models algorithms do not handle large data properly because of missing data normalization.

876167

Minor

Parser

During Testing Rules, parsed event attribute values containing semicolon do not show correctly.

936491

Minor

Performance Monitoring

After cloning a JDBC Performance Object, Test may fail.

1021803, 1004011

Minor

Query

For ClickHouse, there may be result discrepancy between Search on GUI and CSV Export.

996566

Minor

Query

For ClickHouse, it is not possible to do a group by on event attributes with Uint64 values.

1025395

Minor

System

Restarting 'phMonitor' would result in the unnecessarily restarting 'ClickHouseServer'.

1018855

Minor

System

For phClickHouseCSVExport tool, device ip argument should mean reporting ip.

938732

Minor

Upgrade

When collector upgrade fails, phMonitor.pre-upgrade should move back to phMonitor.

1026025

Enhancement

Data work

FortiMail Parser does not parse logs from FortiCloud FortiMail.

1024781

Enhancement

Data work

Win-Security-4688 is not parsing CommandLine in new WinOSXmlParser.

1007504

Enhancement

GUI

Interface Usage Dashboard data cannot be sorted.

996423

Enhancement

GUI

Show Organization names in alphabetical order.

989168

Enhancement

System

Support change_ip operation for worker and non-licensed Supervisor.

Post-Upgrade ClickHouse IP Index Rebuilding

If you are upgrading ClickHouse based deployment from pre-7.1.1 to 7.1.6, then after upgrading to 7.1.6, you need to run a script to rebuild ClickHouse indices. If you are running 7.1.2, 7.1.3, 7.1.4, or 7.1.5, and have already executed the rebuilding steps, then nothing more needs to be done.

For details about this issue, see Release Notes 7.1.3 Known Issue.

The rebuilding steps are available in Release Notes 7.1.4 - Script for Rebuilding/Recreating pre-7.1.1 ClickHouse Database Indices Involving IP Fields.