Fortinet black logo

FortiSIEM Reference Architecture Using ClickHouse

Selecting an Event Database Solution

Selecting an Event Database Solution

When selecting an event database storage system, consider the following factors over the life of the solution:

  • Online event storage requirements

  • Log ingestion rate

  • Large scale query and reporting performance requirements

  • 3rd party application integration requirements

  • Resilience

  • Cost and complexity of deployment, and the ability of the organization to support it

The choice of event database storage can affect the cost, scalability and performance of the solution. Migrating between event database storage architectures is not always possible.

EventDB using NFS

Single node ClickHouse

Multi node ClickHouse

External Elasticsearch[1]

Recommended EPS

1 – 30K EPS

0 - 20K EPS

0 - 1M EPS

0 - 1M EPS

Large query performance

Low - Medium

Medium

High

High

3rd party data access

No

No

No

Yes

Resilience [2]

High – depending on NFS

Low

High

High

Cost and Complexity

Low

Low

Medium

High

Storage Efficiency

High

High

High

Low

[1] Elasticsearch performance and resilience is dependent on the design of the external Elasticsearch cluster and the amount of resources provided to it.

Selecting an Event Database Solution

When selecting an event database storage system, consider the following factors over the life of the solution:

  • Online event storage requirements

  • Log ingestion rate

  • Large scale query and reporting performance requirements

  • 3rd party application integration requirements

  • Resilience

  • Cost and complexity of deployment, and the ability of the organization to support it

The choice of event database storage can affect the cost, scalability and performance of the solution. Migrating between event database storage architectures is not always possible.

EventDB using NFS

Single node ClickHouse

Multi node ClickHouse

External Elasticsearch[1]

Recommended EPS

1 – 30K EPS

0 - 20K EPS

0 - 1M EPS

0 - 1M EPS

Large query performance

Low - Medium

Medium

High

High

3rd party data access

No

No

No

Yes

Resilience [2]

High – depending on NFS

Low

High

High

Cost and Complexity

Low

Low

Medium

High

Storage Efficiency

High

High

High

Low

[1] Elasticsearch performance and resilience is dependent on the design of the external Elasticsearch cluster and the amount of resources provided to it.