What's New in 7.1.1

This release contains the following key enhancements and bug fixes.

• Key Enhancements

• Bug Fixes and Enhancements

• Known Issue

Key Enhancements

• Rocky Linux 8.9

• Redis Memory Usage Optimization

• Support for Trend Vision One

• More SOC Queries via Fortinet Advisor

• SIGMA Rule Fixes

• Public REST API Throttling

Rocky Linux 8.9

This release updates Rocky Linux OS to 8.9 and includes published Rocky Linux OS updates until November 28, 2023. The list of updates can be found at https://errata.rockylinux.org/.

FortiSIEM Rocky Linux Repositories (os-pkgs-cdn.fortisiem.fortinet.com and os-pkgs-r8.fortisiem.fortinet.com) have also been updated to include fixes until July 14, 2023. Therefore, FortiSIEM customers in versions 6.4.1 and above, can upgrade only their Rocky Linux versions by following the procedures described in FortiSIEM OS Update Procedure.

Redis Memory Usage Optimization

FortiSIEM uses Redis to distribute CMDB Group Objects (including Malware IP/Domain/URL/Hash objects) from Supervisor PostGreSQL database to the Worker nodes. A Malware IP or Domain group containing a large number of entries can cause Redis to hit its memory limit and cause Search queries to fail. In this release, by using compression techniques, Redis peak memory usage is reduced significantly. This enables FortiSIEM to handle more threat feed entries, and more CMDB Groups.

As an example, in Fortinet experiments with 1 million FortiGuard Malware IP, 3 million Malware Domains, 500 thousand Malware URL and with App Server Java memory set to 10GB, Redis peak memory usage is reduced from 1GB in 7.0.2 to 156MB in 7.1.1. For this case, compression resulted in more than 80% reduction of Redis peak memory.

Support for Trend Vision One

This release adds support for Trend Vision One XDR platform. See Trend Vision One in the External Systems Configuration Guide for integration details.

More SOC Queries via Fortinet Advisor

Fortinet Advisor recognizes and responds to the following Security Operations Center (SOC) questions.

• Get my FortiSIEM environment

• Get latest 10 high severity Incidents

• Get most frequent 10 Incidents

• Get Top 10 risky users

• Get Top 10 risky devices

SIGMA Rule Fixes

This release updates several FortiSIEM rules adapted from SIGMA rules. Updates involve regular expression conversions from SIGMA format to FortiSIEM format.

Public REST API Throttling

To safeguard Supervisor performance, FortiSIEM now throttles the volume of public REST API requests. Concurrent API requests are limited per source IP and globally. Once limits are reached, error code 429 is sent in response.

The limits are defined in /opt/phoenix/config/phoenix_config.txt:

global_max_concurrent_public_api_requests=50

per_ip_max_concurrent_public_api_requests=10

Notes:

1. When a single source IP makes more than 10 concurrent API calls, then the 11th request will receive a 429 HTTP(S) error code. If the client retries and one or more of its earlier calls finishes and the number of active calls becomes lower than 10, then the new call will succeed.

2. If the total of all active API calls from all sources is over 50, then the 51st request will receive a 429 HTTP(S) error code.

3. When a 429 error code is encountered, API Clients should implement backoff, waiting longer between each subsequent retry.

Bug Fixes and Enhancements

This release contains the following fixes and enhancements.

Bug ID	Severity	Module	Description
971855	Major	App Server	Null pointer exception may occur during App Server incident handling.
971840	Major	App Server	App Server may hit deadlock issue in Postgres during FortiSIEM node health update.
977554	Major	ClickHouse	After upgrading to 7.1.0, adding new ClickHouse node to the same shard fails with DDL error.
914974	Major	Rule Engine	User created security incidents auto-clear after 24 hours even if auto_clear_security_incidents=0 is set.
975345	Minor	App Server	For Windows and Linux Agents, agent monitoring attributes overwrite agentless monitoring attributes, when both agentless methods (such as OMI or SSH) are used along with agents on the same server.
973567	Minor	App Server	After cloning an existing rule and changing the evaluation mode to scheduled, Incidents are still evaluated in streaming mode.
972257	Minor	App Server	Summary Dashboards do not show performance metrics collected by Windows Agent.
971860	Minor	App Server	For Event Receive Hour/Day/Week queries, Query Result Export and Scheduled Report do not work correctly.
971276	Minor	App Server	System defined and user defined Network objects with same IP range become incorrectly linked together.
971126	Minor	App Server	Invalid Query XML for IN queries with more than 1 Individual Countries.
969372	Minor	App Server	Public REST API for Event Query and Archive Query return no events if report syntax is invalid. It should return error instead.
968983	Minor	App Server	Content update fails if there are dashboard widgets in the content update.
968751	Minor	App Server	Box.com integration may cause App Server to lock up when auth token expires.
968266	Minor	App Server	For Incident public REST API, queries for second and subsequent pages may fail with 503 error code if called too fast.
962913	Minor	App Server	Need to throttle public REST API queries by returning HTTP status code 429, when client sends in too many requests.
939273	Minor	App Server	Cannot modify device properties for multi-tenant collector.
936243	Minor	App Server	Timezone selection for Europe/Berlin is not listed in UTC+2, but it is in UTC+1.
927843	Minor	App Server	Discovering a device via FSM Agent and EMS/FGT integration results in duplicate CMDB entries.
926647	Minor	App Server	CMDB Device Report: No result for 'Property Event Receive Time Gap [Low/High] Threshold minutes'.
970594	Minor	ClickHouse Backend	Update phClickHouseImport tool to support event DB data import from custom directory instead of CUSTOMER_1 only.
974846	Minor	Discovery	Test Connectivity for Cisco FireAmp fails.
970075	Minor	Discovery	GitLab discovery failure: Need to use host name as IP does not work during SSL handshake.
931808	Minor	Discovery	For standalone FortiSwitch, Network Interfaces not discovered via SNMP v3 because of lack of support for SHA-224, SHA-256, SHA-384 and SHA-512 for authentication and AES-192 and AES-256 for encryption.
976427	Minor	GUI	Analytics > Investigation page, Run Reports > Event Receive Time column shows epoch value instead of date formatted values.
976046	Minor	GUI	User with Dashboard only role gets empty landing page after login.
974384	Minor	GUI	In CMDB Report, Latest Monitor Time and Latest Event Pulling Time fields show epoch value instead of date formatted values.
972715	Minor	GUI	Check Reputation in Real Time/Historical Search does not work.
971557	Minor	GUI	NullPointerException in the POST SAML response after modifying the idle timeout for Azure SSO user.
966730	Minor	GUI	Name field from External Authentication shouldn't allow 'space' when the protocol is SAML.
966728	Minor	GUI	SAML Organization field for SAML Role configuration doesn't accept space + umlaut characters.
964794	Minor	GUI	For user defined rules/reports, the user cannot move rules or reports to a new custom folder without creating a copy.
963867	Minor	GUI	Malformed IP address can be successfully imported from .CSV file without error checking.
957400	Minor	GUI	CMDB Report - Rule query - Scope attribute only takes integer, but needs string.
927769	Minor	GUI	GUI allows invalid / character to be added in port field for FortiOS credentials.
887630	Minor	GUI	Widget Setting as Single Line Chart and Display Type as Text - COUNT(Matched Events) displays no count.
628705	Minor	GUI	It is better to disable 'Test' button for OKTA authentication policy instead of showing 'IP/Host is required'.
970976	Minor	Parser	In 'PH_SYSTEM_IP_EVENTS_PER_SEC' event, Reporting Device is set incorrectly.
966727	Minor	Parser	For Amazon AWS CloudWatch, CMDB is populated for each discovered device.
974448	Minor	phMonitor	Disaster recovery setup may fail with 1 hour timeout, if CMDB replication takes a long time (resulting from CMDB being large and network bandwidth being slow).
968131	Minor	Query	Query using DevicetoCMDBAttr does not return any result for custom property.
965081	Minor	Report	In PDF Report, legend may not always show.
971810	Minor	System	phziplogs does not pull phoenix-x.log due to file format change from phoenix.log.x.
966773	Minor	System	Collector fresh-install needs internet to uninstall rpcbind.
972752	Minor	Windows Agent	Windows Agent reports "Disk Full" for Optical Drives.
954108	Minor	Windows Agent	Agent can't talk to Collector (verification fails) when Collector has a TLS certificate.
964501	Enhancement	ClickHouse Backend	Generate an incident and system error when free disk of ClickHouse is lower than 20%.
961884	Enhancement	ClickHouse Backend	Enhancement - Procedures for incrementally adding ClickHouse storage.
972486	Enhancement	Data work	Add rule/report for Apache ActiveMQ Ransomware Attack.
971135	Enhancement	Data work	Netflow dashboards do not include all relevant traffic.
967829	Enhancement	Data work	Windows - Need to parse Logon GUID to userID instead of machineGUID.
966160	Enhancement	Data work	Need to enhance FortiEDR Rule and event parsing.
964446	Enhancement	Data work	FortiGate Events generated with logID 0100044545 needs to be parsed as FortiGate-event-admin-delete.
963543	Enhancement	Data work	Missing column 'appServerState' when loading Application Server dashboard.
962882	Enhancement	Data work	Update Carbon Black CEF parser.
939482	Enhancement	Data work	HPiLoParser Unknown Event due to different syslog header format.
936650	Enhancement	Data work	PANOS parser enhancement needed to parse original VM name from Panorama logs.
916555	Enhancement	Data work	'Group Policy Object Created/Modified' rules have the same event type filter.
912298	Enhancement	Data work	Parse device hostname for FortiAuthenticator parser.
869437	Enhancement	Data work	Update Zscaler log integration in JSON format.
850455	Enhancement	Data work	Update KasperskyParser, update RegEx.
964471	Enhancement	Generative AI	In ChatGPT audit log, provide visibility of user and org ID.
969605	Enhancement	Performance Monitoring	mib2xml enhancements to handle Dell iDRAC.
963416	Enhancement	Rule Engine	Sometimes phRuleWorkers drops events, while load is light and has CPU and memory resources.

Known Issue

After upgrading to 7.1.1, in ClickHouse based deployments, searching on IP fields (namely, Reporting IP, Source IP, Destination IP, and Host IP) does not show correct results for events stored prior to upgrade. Events are still stored in ClickHouse, but searches on events before the upgrade do not return results, while searches on events stored after the upgrade work correctly. All other searches work correctly.

This issue is related to a recent change in ClickHouse version 23.3 in how IPV6 fields are represented. See the following URLs for more information.

• https://github.com/ClickHouse/ClickHouse/issues/58885

• https://github.com/ClickHouse/ClickHouse/issues/56487

• https://github.com/ClickHouse/ClickHouse/pull/57707

• https://github.com/ClickHouse/ClickHouse/issues/54162

Workaround

The workaround requires recreating old indices involving Reporting IP, Source IP, Destination IP, and Host IP that were created before the 7.1.1 upgrade. In our experiments, Fortinet has not seen any event loss or FortiSIEM service interruption during this process.

1. Go to root shell by running the following command:

   sudo -s

2. Change directory to /tmp by running the following command:

   cd /tmp

3. Run the following command:

   clickhouse-client

4. Ensure that /data-clickhouse-hot-1 has at least 10% free disk space. This space is required during index re-creation (Step 5 below). If free disk space is less that 10%, then run the following SQL command (4a.) to get the list of oldest ClickHouse partitions residing on the /data-clickhouse-hot-1 disk and either move them to another disk or tier, or delete them until /data-clickhouse-hot-1 has at least 10% free disk space. These commands need to be run only on ALL data nodes in every shard. The first command (4a.), identifies the largest partitions on the /data-clickhouse-hot-1 disk. The remaining commands enable you to move the data to another tier (4b.), or another disk (4c.), or delete the data (4d.).

   1. Identify the largest ClickHouse partitions in Hot node:

      SELECT disk_name, partition, extract(partition, '\(\d+,(\d+)\)') as date, formatReadableSize(sum(bytes_on_disk)), formatReadableSize(sum(data_uncompressed_bytes)) FROM system.parts WHERE (table = 'events_replicated') AND path LIKE '%hot-1%' AND active GROUP BY disk_name, partition ORDER BY disk_name ASC, date ASC limit 10

   2. Move the data to another tier:
      

      ALTER TABLE fsiem.events_replicated MOVE PARTITION <partition expression from (a) > TO VOLUME <next tier>

   3. Move the data to another disk:

      ALTER TABLE fsiem.events_replicated MOVE PARTITION <partition expression from (a) > TO disk <another disk>

   4. Delete the data:

      ALTER TABLE fsiem.events_replicated DROP PARTITION <partition expression from (a) >

      Example:
      

      Output from command in 4a.:
      

      

      To move the first partition (size 3.98 GiB) to Warm tier, issue the following command as shown in 4b.

      ALTER TABLE fsiem.events_replicated MOVE PARTITION (18250, 20240115) TO VOLUME 'warm'

      To move the first partition (size 3.98 GiB) to another disk in Hot tier, issue the following command as shown in 4c.

      ALTER TABLE fsiem.events_replicated MOVE PARTITION (18250, 20240116) TO disk 'data_clickhouse_hot_2'

      To delete the first partition (size 3.98 GiB), issue the following command as shown in 4d.

      ALTER TABLE fsiem.events_replicated DROP PARTITION (18250, 20240116)

5. Run the following commands sequentially. This will drop/add/recreate all affected indices: Reporting IP, Source IP, Destination IP, and Host IP within ClickHouse. These commands need to be run only on one data node per shard. Note that the first command (drop) in every index may take some time to complete. User must wait until the command completes before issuing the next command.

   alter table fsiem.events_replicated drop index index_reptDevIpAddr_bloom_filter
   alter table fsiem.events_replicated add INDEX index_reptDevIpAddr_bloom_filter reptDevIpAddr TYPE bloom_filter GRANULARITY 5 AFTER index_customer_set
   alter table fsiem.events_replicated materialize index index_reptDevIpAddr_bloom_filter
   
   alter table fsiem.events_replicated drop index index_srcIpAddr_bloom_filter
   alter table fsiem.events_replicated add INDEX index_srcIpAddr_bloom_filter metrics_ip.value[indexOf(metrics_ip.name, 'srcIpAddr')] TYPE bloom_filter GRANULARITY 5 AFTER collectorId_set
   alter table fsiem.events_replicated materialize index index_srcIpAddr_bloom_filter
   
   alter table fsiem.events_replicated drop index index_destIpAddr_bloom_filter
   alter table fsiem.events_replicated add INDEX index_destIpAddr_bloom_filter metrics_ip.value[indexOf(metrics_ip.name, 'destIpAddr')] TYPE bloom_filter GRANULARITY 5 AFTER index_srcIpAddr_bloom_filter
   alter table fsiem.events_replicated materialize index index_destIpAddr_bloom_filter
   
   alter table fsiem.events_replicated drop index index_hostIpAddr_bloom_filter
   alter table fsiem.events_replicated add INDEX index_hostIpAddr_bloom_filter metrics_ip.value[indexOf(metrics_ip.name, 'hostIpAddr')] TYPE bloom_filter GRANULARITY 5 AFTER index_user_bloom_filter
   alter table fsiem.events_replicated materialize index index_hostIpAddr_bloom_filter