Appendix: Sizing

Before you start your setup, you need to determine the storage requirements for the portal and collector databases. To do this, determine the approximate values for the following:

• Expected log rate (logs per second)
• Number of customers
• Number of days to retain data
• Number of FortiGate devices

The values are based on one VDOM per customer and an 80/20 storage ratio of the portal database to the collector database. If you are using FortiGate HA, count only the number of HA masters for the number of FortiGate devices.

The expected log rate is the overall value for logs to the collector. To find the number of logs per second for a VDOM based on the last seven days of logs, use the diagnose test application command.

For example:

FG1K5D3I14801425 # diagnose test application miglogd 4
info for vdom: root
memory
traffic: logs=63016358 len=34499571723, Sun=0 Mon=0 Tue=63016358 Wed=0 Thu=0 Fri=0 Sat=0
event: logs=2756 len=972616, Sun=324 Mon=324 Tue=345 Wed=324 Thu=740 Fri=375 Sat=324

The example shows the log counters for a seven-day period and gives the total number of entries for each log type for each VDOM.

To calculate the number of log entries per second, take the sum of the logs (for example, 63,016,358 + 2,756 = 63,019,114), divide by 7 (for example, 63,019,114/7 = 9,002,731), and then divide by 86,400 (for example, 9,002,731/86,400 = 104) to get the number of logs per second.

A single collector instance (collector and database) can handle 15,000 logs per second. The portal supports multiple collectors on multiple VDOMs to increase the log rate and storage.

The following calculations can be used to determine the storage values for the collector and the portal and collector databases:

• Syslog /second /customer = expected log rate / number of customers
• Portal database size in MB for one syslog /second /day size = 0.156
• Portal database size in GB /customer /day = (syslog /second /customer * 0.156) / 1024
• Portal database size /customer to retain data = portal database size in GB /customer /day * number of retention days
• Minimum customer storage = portal database size /customer to retain data / (0.8 * 0.2) (The minimum value is 5 GB.)
• Collector disk size = 2.65 + (number of FortiGate units * 3.25) (The minimum value is 80 GB.)
• Portal database disk size = minimum customer storage * number of customers * 1.25 * 0.2 + 20
• Collector database disk size = minimum customer storage * number of customers * 1.25 * 0.8 + 20

Values in boldface are either buffer values or allocations for system files.

For example:

Expected log rate	500
Number of customers	5
Number of retention days	60
Number of FortiGate units	10

When you substitute these values into the calculations:

• Syslog /second /customer = 500 / 5 = 100
• Portal database size in GB /customer /day = (100 * 0.156)/1024 = 0.015
• Portal database size /customer to retain data = 0.015 * 60 = 0.914
• Minimum customer storage = 0.914 / (0.8 * 0.2) = 5.712 ~ 6 GB
• Collector disk size = 2.65 + (10 * 3.25) = 35.15 GB, so use the minimum value of 80 GB
• Portal database disk size = 6 * 5 * 1.25 * 0.2 + 20 = 27.5 GB
• Collector database disk size = 6 * 5 * 1.25 * 0.8 + 20 = 50 GB

Configuring the collector VM disk size

The NTP source must be the same for all portal and collector VMs to synchronize the log time stamps across all devices.

When you deploy the OVF image for the first time, you can configure the disk size. There are two virtual disks. Hard disk 1 is the flash disk (2 GB); hard disk 2 is the storage disk (80 GB by default.). You can increase the size of the storage disk when deploying the VM. See the following figure. (Do not increase the size of hard disk 1.)

When the collector is running, use the execute lvm info command to see the disk size.

If you have already deployed the collector VM, you can add an additional disk.

To add a disk:

1. Shut down the collector.
2. Edit the VM properties.
3. Add an additional disk. See the following figure.
   
   
4. Start the collector.
5. Use the lvm info command again. You should now see an additional, unused disk.
6. Enter the execute lvm extend command to use the new hard disk. The collector will reboot.
   
   
7. When the collector is running, use the execute lvm info command to see the new disk size.