Disaster Recovery Operations
Primary (Site 1) Down, Secondary (Site 2) becomes Primary
Primary (Site 1) has failed and unavailable. The current Secondary (Site 2) needs to become Primary. The Collectors will start buffering events since the Site 1 Workers are down.
Follow the steps below to promote Site 2 to Primary.
Log on to Site 2 (Secondary) as root and run the following command:
phsecondary2primary
Site 2 (Secondary) becomes Primary, and you can log on to Site 2, which becomes the current Primary, to continue your work.
Note: The process should take approximately 10 minutes to complete. Both FortiSIEM nodes become independent after running the command.
Site 1 Up and Supervisor / Worker Recovered
Site1 has recovered after failure meaning that the Supervisor and Workers are up. You first need to make Site 1 as Secondary and then (optionally) switch roles if you want Site 1 to become Primary again.
Follow the instructions below to make Site 1 as Secondary.
-
Logon to the current Primary FortiSIEM node (this should be Site 2) using the GUI.
-
Navigate to ADMIN > License > Nodes.
-
Select the Secondary FortiSIEM node listed (this should be Site 1). It should appear as Inactive under the Replication Status column.
-
Click Edit.
-
Review the information to ensure that all the information is correct.
Note: The information is read only. -
When done, click Save. The Original Primary (Site 1) now becomes the Secondary in your Disaster Recovery configuration. The Replication Status changes from Inactive to Active.
Now, if you want to switch roles so Site 1 becomes Primary again, follow the instructions in Switching Primary and Secondary Roles.
Site 1 Up but Supervisor / Worker Cannot be Recovered
In this case, Site 1 is up, but the Supervisor and Workers cannot be recovered after failure.
In this situation, you need to first reinstall a Supervisor and Workers on Site 1 and make Site 1 as Secondary.
-
Logon to the current Primary FortiSIEM node (Site 2) using the GUI.
-
Navigate to ADMIN > License > Nodes.
-
Select the Secondary FortiSIEM node listed. It should appear with as Inactive under the Replication Status column.
-
Click Delete to remove it from the Disaster Recovery configuration.
-
Log out from Site 2.
-
Re-install Supervisor and Workers in Site 1.
-
Log back onto Site 2 GUI.
-
Add Site 1 as a new Secondary by following the instructions in Configuring Disaster Recovery.
Now, if you want to switch roles so Site 1 becomes Primary again, follow the instructions in Switching Primary and Secondary Roles.
Switching Primary and Secondary Roles
If you need to change your Disaster Recovery setup so that Site 2 will be Secondary, and Site 1 will be Primary, take the following steps.
-
Logon to the current Secondary FortiSIEM node (Site 1) as root, and run the following command:
phsecondary2primary
When the job is completed, Site 1 is now the Primary.
-
Logon to the Site 1 (Primary) UI.
-
Navigate to ADMIN > License > Nodes.
-
Select the Site 2 (Secondary) FortiSIEM node listed and click Edit.
-
Review the information to ensure that all the information is correct.
-
When done, click Save.
Site 1 will become Primary and Site 2 will be Secondary. Remember to change the DNS addresses after this role switch so that users are logging on to the Primary and Collectors are sending to Primary Workers.
Recovering from Human Error
If, by mistake, the phsecondary2primary command is executed, it turns Site 2 (Secondary) node to Primary. At this point, you have two independent Primary nodes. To recover, take the following steps:
-
Logon to the FortiSIEM you wish to be the current Primary.
-
Navigate to ADMIN > License > Nodes.
-
Select the Secondary node.
-
Click Edit.
-
Review the information to ensure that all the information is correct.
-
When done, click Save.
Upgrading with Disaster Recovery Enabled
To upgrade your FortiSIEMs in a Disaster Recovery environment, take the following steps.
-
Upgrade the Primary Supervisor and Workers
-
After the Primary is fully upgraded, upgrade the Secondary Supervisor and Workers.
After Step 1, the Secondary Supervisor database schema is already upgraded. Step 2 simply upgrades the executables in Site 2.
For information on upgrading, see the Upgrade Guide.
Turning Off Disaster Recovery
To turn off Disaster Recovery, take the following steps.
-
Logon to the current Primary GUI.
-
Navigate to ADMIN > License > Nodes.
-
Select the Secondary FortiSIEM node listed and click Delete.
-
Click Yes to confirm the operation.