Provisioning the Event Database
Event Database Options and Scalability
FortiSIEM combines four back-end databases into a single GUI for a seamless user experience. The databases are:
|
Profile Database |
Stores risk data and system data |
|
SVN |
Stores device configuration data |
|
CMDB |
Stores data relating to the FortiSIEM internal CMDB |
|
Event Database |
Event storage |
The profile database, SVN and CMDB are hosted on the Supervisor node in all deployment types. They are self-managing and need minimal consideration from a design and deployment perspective other than to provision the supervisor with adequate resources and high performance flash storage.
The event database design must be considered in all deployments. FortiSIEM supports several event data storage options:
Only one event storage option can be configured at any one time. It is possible to reconfigure the system to use a different event storage in-life but migrating data can be difficult or impossible. Choose the best storage solution for the life of the solution from the outset. Each database option is explained in turn below:
FortiSIEM EventDB
FortiSIEM EventDB is the traditional event storage solution for FortiSIEM deployments. It uses a proprietary database structure that can be hosted on a local disk in a small single-server deployment, or on a high performance NFS server for larger distributed deployments. The EventDB is easy to use but does not offer the scalability of ClickHouse. It is suitable for smaller deployments of up to approximately 30k EPS.
FortiSIEM Integrated ClickHouse
The ClickHouse database is integrated into FortiSIEM from release 6.5.0, with scale-out supported from release 6.6.0. Many customers will find the ClickHouse event storage to be the optimal choice, as it is integrated into the solution and provides very high performance and scalability. Most of this document assumes the use of ClickHouse.
FortiSIEM with ClickHouse runs the ClickHouse database across nodes in the FortiSIEM cluster. In a small single server solution, the FortiSIEM Supervisor runs the FortiSIEM application and the ClickHouse database services. In a larger multi-node solution, the ClickHouse database runs on separate FortiSIEM worker nodes, with the event data being stored on an additional local disk in each Worker node. These worker nodes run both the distributed ClickHouse database and the FortiSIEM distributed correlation and query engine in a single unified architecture.
A multi-node solution benefits from the scalability and resilience offered by ClickHouse. Multiple shards can be configured for increased performance. Each shard can have multiple replicas for increased resilience.
The number of shards required varies depending on the EPS volume and resilience requirements. The amount of storage required on each FortiSIEM node depends on the log size and retention requirements. Scale-out ClickHouse also requires keeper nodes, which manage the write operations across the distributed database architecture.
External Elasticsearch Cluster
FortiSIEM can also be configured to store event data on an external Elasticsearch cluster. Elasticsearch is a scalable, distributed, high performance database that some customers may prefer for large deployments. If using the Elasticsearch event storage option, then a separate Elasticsearch cluster must be designed and deployed on separate hardware; Elasticsearch is not provided as part of FortiSIEM, and does not run directly on FortiSIEM nodes.