FortiAnalyzer HA graceful upgrade

With this new feature, FortiAnalyzer HA supports graceful upgrading to avoid log loss, and also allow a trial period of the new image and support roll-back to the existing firmware if the new image has any issues.

In the following example, FortiAnalyzer HA is being upgraded from version 6.4.5 to 7.0.0, and contains two members: FAZ-1 is the Primary and FAZ-2 is the Secondary.

Graceful upgrade is only supported when upgrading from 6.4.5 and above to version 7.0.0 and later. Data conversion depends on each application using this framework to do the conversion between the different versions. In 7.0.0, only incident conversion is supported when syncing from a higher level version to lower level version.

To update FortiAnalyzer HA gracefully:

1. Upgrade FAZ-2, the Secondary unit, to the new version. In this example, FAZ-2 is upgraded from version 6.4.5 to 7.0.0.
2. After FAZ-2 is up and running, check that there are no critical crashes and that the Primary can still forward logs to the Secondary. Event Alert and Incidents still can be received from 6.4.5 to 7.0.0.
   • To check that there are no critical crashes, use the following CLI command:

     diagnose debug crash read

   • To check that logs are being forwarded, use the following CLI command:

     diagnose test application logfwd 4

   • To create an example to check event alert synchronization, log in to FortiAnalyzer using the wrong password to generate a new local event log, and check on both FAZ-1 and FAZ-2 to confirm that the event alert can be found on both devices.
     
   • To create an example to check incident synchronization, create a new incident on FAZ-1, then check on FAZ-2 to see if it was correctly synced across devices.
     
3. Since both devices are now running on different firmware versions, configuration synchronization is unavailable at this time. You can check this setting using the diagnose ha status command in the FortiAnalyzer CLI.
   In this example, the config sync status is down, and no configuration changes can be synced from the Primary to Secondary unit.

   diagnose ha status

   HA-Status: Primary

   up-time: 11h38m12.811s

   config-sync: Allow

   serial-no: FL-1KE3R16000432

   fazuid: 2626920937

   hostname: FAZ1000E-2

   HA-Secondary HA1000e@192.168.1.90 FL-1KE3R16000419

   ip: 192.168.1.90

   serial-no: FL-1KE3R16000119

   fazuid: 1239922567

   hostname: FAZ1000E

   conn-st: up

   up/down-time: 11h38m10.455s

   conn-msg: firmware version mismatch (v6.4.6-build2363 210531 (GA))

   cfgsync-st: down

   data-init-sync-st: done, 11h37m49.396s

4. Once FAZ-2 is in data-sync with FAZ-1, an administrator can trigger HA-failover using the CLI to switch FAZ-2 to the Primary role.
   1. In the FortiAnalyzer CLI, enter the command diagnose ha failover to make FAZ-2 the Primary.

      FAZ1000E-1 # diagnose ha failover

   2. Use the command diagnose ha status to confirm the role of FAZ-2 as the new Primary.

      FAZ1000E-2 # diagnose ha status

      HA-Status: Primary

      up-time: 11h38m12.811s

      config-sync: Allow

      serial-no: FL-1KE3R16000432

      fazuid: 2626920937

      hostname: FAZ1000E-2

      HA-Secondary HA1000e@192.168.1.90 FL-1KE3R16000419

      ip: 192.168.1.90

      serial-no: FL-1KE3R16000119

      fazuid: 1239922567

      hostname: FAZ1000E

      conn-st: up

      up/down-time: 11h38m10.455s

      conn-msg: firmware version mismatch (v6.4.6-build2363 210531 (GA))

      cfgsync-st: down

      data-init-sync-st: done, 11h37m49.396s

5. Now is the time for the administrator to try out the new image on FAZ-2.
   As part of the graceful upgrade, logs can still be forwarded from a higher version (7.0.0) to a lower version (6.4.5) without issue, and incidents are synched from the higher version (7.0.0) to the Secondary running a lower version (6.4.5).
   During this time you should avoid any configuration changes, as they will not be synchronized between versions.
6. Check the upgrade guide checklist to confirm the new Primary is working as expected.
   After a few hours or a day, FAZ-1 can be upgraded to the new firmware version (7.0.0). After FAZ-1 is upgraded, FAZ-2 will continue to operate as the Primary. You can failover again to return FAZ-1 to operating as the Primary, or keep FAZ-2 as the new Primary.