Scaling FortiSIEM with ClickHouse In-Life

Fortinet recommends designing the solution to have the required number of nodes and shards to meet the maximum anticipated EPS from day one. FortiSIEM with ClickHouse is very efficient; just a few nodes can ingest very high EPS. In many cases, organizations can significantly simplify the in-life administration of the solution by deploying the higher number of nodes initially, rather than adding more later.

Scaling performance in-life by increasing the number of shards is possible, but is a little more complex. When a new shard is added to the cluster, the system will start to balance both data storage and queries across all of the shards to increase performance, but the old data stored in the existing shards is not rebalanced into the new shard. This means that: (1) queries against the old data will only be serviced by old shards, although the query is sent to old and new shards; and (2) new data is written to all shards so the old shards will continue to purge old data as new data is ingested, even though the new shard has spare storage. Over time, this situation will resolve itself as the old data is fully purged from the existing shards.

Over time, the disk allocated to event storage within a shard may become full. Scaling storage to increase event storage capacity is straightforward in a virtual appliance based solution - simply assign more storage to the virtual machine (VM) by extending the disk hosting the ClickHouse database on the nodes within the shards.