Medium and Large Deployments with Replicas

Medium and large deployments require scalability and resilience. Recall that the number of shards affects database performance, and the number of replicas affects resilience. Many organizations will find that two replicas per shard meets their resilience requirements, so the main decisions are: (1) the number of shards; and (2) the ClickHouse keeper architecture.

A generic medium size deployment is shown below. It has one shard and two replicas. This type of deployment can handle up to 20k EPS.

• The Supervisor node runs the keeper process and one of the ClickHouse replicas

• The Worker node runs the second ClickHouse replica

• Additional collector nodes are deployed on the edge for flexibility, scalability, and to support FortiSIEM agents.

A generic larger deployment is shown below. This type of deployment can handle up to 1M EPS with sufficient shards and resources.

• One or more dedicated nodes are deployed as the FortiSIEM Supervisor nodes.

• Three worker nodes are deployed as dedicated keeper nodes.

• ‘n’ shards are deployed for data processing. Refer to the ClickHouse Sizing Guide for the suggested number of shards to support a specific EPS requirement.

• Each shard has two nodes, one for replica #1, and one for replica #2.

  • Additional replicas can be designed into the solution if required.

• Additional collector nodes are deployed on the edge for flexibility, scalability, and to support FortiSIEM agents.