Fortinet black logo

FortiGate-7000 Handbook

Installing firmware on individual FIMs and FPMs

Copy Link
Copy Doc ID bf67d868-679e-11e9-81a4-00505692583a:535892
Download PDF

Installing firmware on individual FIMs and FPMs

You can install firmware on individual FIMs or FPMs by logging into the FIM or FPM GUI or CLI. You can also setup a console connection to the FortiGate-7000 front panel Management Module and install firmware on individual FIMs or FPMs from a TFTP server after interrupting the FIM or FPM boot up sequence from the BIOS.

Normally you wouldn't need to upgrade the firmware on individual FIMs or FPMs because the FortiGate-7000 keeps the firmware on all of the FIMs and FPMs synchronized. However, FIM or FPM firmware may go out of sync in the following situations:

  • Communication issues during a normal FortiGate-7000 firmware upgrade.
  • Installing a replacement FIM or FPM that is running a different firmware version.
  • Installing firmware on or formatting an FIM or FPM from the BIOS.

To verify the firmware versions on each FIM or FPM you can check individual FIM and FPM GUIs or enter the get system status command from each FIM or FPM CLI. You can also use the diagnose sys confsync status | grep in_sy command to see if the FIMs and FPMs are all synchronized. In the command output, in_sync=1 means the FIM or FPM is synchronized. In_sync=0 means the FIM or FPM is not synchronized, which could indicated the FIM or FPM is running a different firmware build than the primary FIM.

The procedures in this section work for FIMs or FPMs in a standalone FortiGate-7000. These procedures also work for FIMs or FPMs in the primary FortiGate-7000 in an HA configuration. To upgrade firmware on an FIM or FPM in the backup FortiGate-7000 in an HA configuration, you should either remove the backup FortiGate-7000 from the HA configuration or cause a failover so that the backup FortiGate-7000 becomes the primary FortiGate-7000.

In general, if you need to update both FIMs and FPMs in the same FortiGate-7000, you should update the FIMs first as the FPMs can only communicate through FIM interfaces.

Upgrading FIM firmware

Use the following procedure to upgrade the firmware running on a single FIM. For this procedure to work, you must connect at least one of the FIM MGMT interfaces to a network. You must also be able to log in to the FIM GUI or CLI from that MGMT interface. If you perform the firmware upgrade from the CLI, the FIM must be able to communicate with an FTP or TFTP server.

During the upgrade, the FIM will not be able to process traffic. However, the other FIM and the FPMs should continue to operate normally.

  1. Log into the FIM GUI or CLI and perform a normal firmware upgrade.
    You may need to use the special port number to log in to the FIM in slot two (for example, browse to https://192.168.1.99:44302).

  2. Once the FIM restarts, verify that the new firmware has been installed.

    You can do this from the FIM GUI dashboard or from the FIM CLI using the get system status command.

  3. Verify that the configuration has been synchronized to the upgraded FIM. The following command output shows the synchronization status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command.If this does not solve the problem, contact Fortinet Support.

    The example output also shows that the uptime of the FIM in slot 2 is lower than the uptime of the other modules, indicating that the FIM in slot 2 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FIM has completely restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

Upgrading FPM firmware

Use the following procedure to upgrade the firmware running on an individual FPM. To perform the upgrade, you must enter a command from the primary FIM CLI to allow ELBC communication with the FPM. Then you can just log in to the FPM GUI or CLI and perform the firmware upgrade.

During this procedure, the FPM will not be able to process traffic. However, the other FPMs and the FIMs should continue to operate normally.

After verifying that the FPM is running the right firmware, you must log back into the primary FIM CLI and return the FPM to normal operation.

  1. Log in to the primary FIM CLI and enter the following command:

    diagnose load-balance switch set-compatible <slot> enable elbc

    Where <slot> is the number of the FortiGate-7000 slot containing the FPM to be upgraded.

  2. Log in to the FPM GUI or CLI using its special port number (for example, for the FPM in slot 3, browse to https://192.168.1.99:44303 to connect to the GUI) and perform a normal firmware upgrade of the FPM.

  3. After the FPM restarts, verify that the new firmware has been installed.

    You can do this from the FPM GUI dashboard or from the FPM CLI using the get system status command.

  4. Verify that the configuration has been synchronized. The following command output shows the sync status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command. If this does not solve the problem, contact Fortinet Support.

    The command output also shows that the uptime of the FPM in slot 4 is lower than the uptime of the other modules, indicating that the FPM in slot 4 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FIM has completely restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

  5. Once the FPM is operating normally, log back in to the primary FIM CLI and enter the following command to reset the FPM to normal operation:

    diagnose load-balance switch set-compatible <slot> disable

    Configuration synchronization errors will occur if you do not reset the FPM to normal operation.

Upgrading FIM firmware from the FIM BIOS using a TFTP server

Use the following procedure to upload firmware from a TFTP server to an FIM. The procedure involves creating a connection between the TFTP server and one of the FIM MGMT interfaces. You don't have to use a MGMT interface on the FIM that you are upgrading.

This procedure also involves connecting to the FIM CLI using a FortiGate-7000 front panel Management Module console port. From the console session, the procedure describes how to restart the FIM, interrupting the boot process, and follow FIM BIOS prompts to install the firmware.

During this procedure, the FIM will not be able to process traffic. However, the other FIM and the FPMs should continue to operate normally.

  1. Set up a TFTP server and copy the firmware file to the TFTP server default folder.

  2. Set up your network to allow traffic between the TFTP server and one of the FIM MGMT interfaces.

    If the MGMT interface you are using is one of the MGMT interfaces connected as a LAG to a switch, you must shutdown or disconnect all of the other connections in the LAG from the switch. This includes the MGMT interfaces in the other FIM.

  3. Using the console cable supplied with your FortiGate-7000, connect the management module Console 1 port on the FortiGate-7000 to the RS-232 port on your management computer.

  4. Start a terminal emulation program on the management computer. Use these settings:

    Baud Rate (bps) 9600, Data bits 8, Parity None, Stop bits 1, and Flow Control None.

  5. Press Ctrl-T to enter console switch mode.

  6. Repeat pressing Ctrl-T until you have connected to the FIM to be updated. Example prompt for the FIM in slot 2:

    <Switching to Console: FIM02 (9600)>

  7. Optionally log in to the FIM's CLI.

  8. Reboot the FIM.

    You can do this using the execute reboot command from the CLI or by pressing the power switch on the FIM front panel.

  9. When the FIM starts up, follow the boot process in the terminal session, and press any key when prompted to interrupt the boot process.

  10. To set up the TFTP configuration, press C.

  11. Use the BIOS menu to set the following. Change settings only if required.

    [P]: Set image download port: MGMT1 (the connected MGMT interface.)

    [D]: Set DHCP mode: Disabled

    [I]: Set local IP address: The IP address of the MGMT interface that you want to use to connect to the TFTP server. This address must not be the same as the FortiGate-7000 management IP address and cannot conflict with other addresses on your network.

    [S]: Set local Subnet Mask: Set as required for your network.

    [G]: Set local gateway: Set as required for your network.

    [V]: Local VLAN ID: Should be set to <none>. (use -1 to set the Local VLAN ID to <none>.)

    [T]: Set remote TFTP server IP address: The IP address of the TFTP server.

    [F]: Set firmware image file name: The name of the firmware image file that you want to install.

  12. To quit this menu, press Q.

  13. To review the configuration, press R.
    To make corrections, press C and make the changes as required. When the configuration is correct, proceed to the next step.

  14. To start the TFTP transfer, press T.

    The firmware image is uploaded from the TFTP server and installed on the FIM. The FIM then restarts with its configuration reset to factory defaults. After restarting, the FIM configuration is synchronized to match the configuration of the primary FIM. The FIM restarts again and can start processing traffic.

  15. Once the FIM restarts, verify that the correct firmware is installed.

    You can do this from the FIM GUI dashboard or from the FPM CLI using the get system status command.

  16. Verify that the configuration has been synchronized.

    The following command output shows the sync status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command. If this does not solve the problem, contact Fortinet support.

    The command output also shows that the uptime of the FIM in slot 2 is lower than the uptime of the other modules, indicating that the FIM in slot 2 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FIM has restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

Upgrading FPM firmware from the FPM BIOS using a TFTP server

Use the following procedure to upload firmware from a TFTP server to an FPM. To perform the upgrade, you must enter a command from the primary FIM CLI to allow the FPM BIOS to communicate through an FIM MGMT interface. The procedure involves creating a connection between the TFTP server and one of the FIM MGMT interfaces.

This procedure also involves connecting to the FPM CLI using a FortiGate-7000 front panel Management Module console port, rebooting the FPM, interrupting the boot from the console session, and following FPM BIOS prompts to install the firmware.

During this procedure, the FPM will not be able to process traffic. However, the other FPMs and the FIMs should continue to operate normally.

After you verify that the FPM is running the right firmware, you must log back in to the primary FIM CLI and return the FPM to normal operation.

  1. Set up a TFTP server and copy the firmware file into the TFTP server default folder.

  2. Log into to the primary FIM CLI and enter the following command:
    diagnose load-balance switch set-compatible <slot> enable bios

    Where <slot> is the number of the FortiGate-7000 slot containing the FPM to be upgraded.

  3. Set up your network to allow traffic between the TFTP server and a MGMT interface of one of the FIMs.

    You can use any MGMT interface of either of the FIMs. If the MGMT interface you are using is one of the MGMT interfaces connected as a LAG to a switch you must shutdown or disconnect all of the other connections in the LAG from the switch. This includes the MGMT interfaces in the other FIM.

  4. Using the console cable supplied with your FortiGate-7000, connect the management module Console 1 port on the FortiGate-7000 to the RS-232 port on your management computer.

  5. Start a terminal emulation program on the management computer. Use these settings:

    Baud Rate (bps) 9600, Data bits 8, Parity None, Stop bits 1, and Flow Control None.

  6. Press Ctrl-T to enter console switch mode.

  7. Repeat pressing Ctrl-T until you have connected to the module to be updated. Example prompt:
    <Switching to Console: FPM03 (9600)>

  8. Optionally log into the FPM's CLI.

  9. Reboot the FPM.

    You can do this using the execute reboot command from the FPM's CLI or by pressing the power switch on the FPM front panel.

  10. When the FPM starts up, follow the boot process in the terminal session and press any key when prompted to interrupt the boot process.

  11. To set up the TFTP configuration, press C.

  12. Use the BIOS menu to set the following. Change settings only if required.

    [P]: Set image download port: MGMT1 (the connected MGMT interface).

    [D]: Set DHCP mode: Disabled.

    [I]: Set local IP address: The IP address of the MGMT interface that you want to use to connect to the TFTP server. This address must not be the same as the FortiGate-7000 management IP address and cannot conflict with other addresses on your network.

    [S]: Set local Subnet Mask: Set as required for your network.

    [G]: Set local gateway: Set as required for your network.

    [V]: Local VLAN ID: Should be set to <none>. (use -1 to set the Local VLAN ID to <none>.)

    [T]: Set remote TFTP server IP address: The IP address of the TFTP server.

    [F]: Set firmware image file name: The name of the firmware image file that you want to install.

  13. To quit this menu, press Q.

  14. To review the configuration, press R.

    To make corrections, press C and make the changes as required. When the configuration is correct proceed to the next step.

  15. To start the TFTP transfer, press T.

    The firmware image is uploaded from the TFTP server and installed on the FPM. The FPM then restarts with its configuration reset to factory defaults. After restarting, the FPM configuration is synchronized to match the configuration of the primary FPM. The FPM restarts again and can start processing traffic.

  16. Once the FPM restarts, verify that the correct firmware is installed.

    You can do this from the FPM GUI dashboard or from the FPM CLI using the get system status command.

  17. Verify that the configuration has been synchronized.

    The following command output shows the sync status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command . If this does not solve the problem, contact Fortinet Support.

    The command output also shows that the uptime of the FPM in slot 4 is lower than the uptime of the other modules, indicating that the FPM in slot 4 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FPM has restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

  18. Once the FPM is operating normally, log back in to the primary FIM CLI and enter the following command to reset the FPM to normal operation:
    diagnose load-balance switch set-compatible <slot> disable
    Configuration synchronization errors will occur if you do not reset the FPM to normal operation.

Synchronizing FIMs and FPMs after upgrading the primary FIM firmware from the BIOS

After you install firmware on the primary FIM from the BIOS after a reboot, the firmware version and configuration of the primary FIM will most likely be not be synchronized with the other FIMs and FPMs. You can verify this from the primary FIM CLI using the diagnose sys confsync status | grep in_sy command. The in_sync=0 entries in the following example output show that the management board (serial number ending in 10) is not synchronized with the other FIM and the FPMs shown in the example.

diagnose sys confsync status | grep in_sy
FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=0
FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=0
FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
...

You can also verify synchronization status from primary FIM Security Fabric dashboard widget.

To re-synchronize the FortiGate-7000, which has the effect of resetting the other FIM and the FPMs, re-install firmware on the primary FIM.

Note You can also manually install firmware on each individual FIM and FPM from the BIOS after a reboot. This manual process is just as effective as installing the firmware for a second time on the primary FIM to trigger synchronization to the FIM and the FPMs, but takes much longer.
  1. Log into the primary FIM GUI.

  2. Install a firmware build on the primary FIM from the GUI or CLI. The firmware build you install on the primary FIM can either be the same firmware build or a different one.

    Installing firmware synchronizes the firmware build and configuration from the primary FIM to the other FIM and the FPMs.

  3. Check the synchronization status from the Security Fabric dashboard widget or using the diagnose sys confsync status | grep in_sy command. The following example ForGate-7040E shows that the primary FIM is synchronized with the other FIM and all of the FPMs because each line includes in_sync=1:

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

Installing firmware on individual FIMs and FPMs

You can install firmware on individual FIMs or FPMs by logging into the FIM or FPM GUI or CLI. You can also setup a console connection to the FortiGate-7000 front panel Management Module and install firmware on individual FIMs or FPMs from a TFTP server after interrupting the FIM or FPM boot up sequence from the BIOS.

Normally you wouldn't need to upgrade the firmware on individual FIMs or FPMs because the FortiGate-7000 keeps the firmware on all of the FIMs and FPMs synchronized. However, FIM or FPM firmware may go out of sync in the following situations:

  • Communication issues during a normal FortiGate-7000 firmware upgrade.
  • Installing a replacement FIM or FPM that is running a different firmware version.
  • Installing firmware on or formatting an FIM or FPM from the BIOS.

To verify the firmware versions on each FIM or FPM you can check individual FIM and FPM GUIs or enter the get system status command from each FIM or FPM CLI. You can also use the diagnose sys confsync status | grep in_sy command to see if the FIMs and FPMs are all synchronized. In the command output, in_sync=1 means the FIM or FPM is synchronized. In_sync=0 means the FIM or FPM is not synchronized, which could indicated the FIM or FPM is running a different firmware build than the primary FIM.

The procedures in this section work for FIMs or FPMs in a standalone FortiGate-7000. These procedures also work for FIMs or FPMs in the primary FortiGate-7000 in an HA configuration. To upgrade firmware on an FIM or FPM in the backup FortiGate-7000 in an HA configuration, you should either remove the backup FortiGate-7000 from the HA configuration or cause a failover so that the backup FortiGate-7000 becomes the primary FortiGate-7000.

In general, if you need to update both FIMs and FPMs in the same FortiGate-7000, you should update the FIMs first as the FPMs can only communicate through FIM interfaces.

Upgrading FIM firmware

Use the following procedure to upgrade the firmware running on a single FIM. For this procedure to work, you must connect at least one of the FIM MGMT interfaces to a network. You must also be able to log in to the FIM GUI or CLI from that MGMT interface. If you perform the firmware upgrade from the CLI, the FIM must be able to communicate with an FTP or TFTP server.

During the upgrade, the FIM will not be able to process traffic. However, the other FIM and the FPMs should continue to operate normally.

  1. Log into the FIM GUI or CLI and perform a normal firmware upgrade.
    You may need to use the special port number to log in to the FIM in slot two (for example, browse to https://192.168.1.99:44302).

  2. Once the FIM restarts, verify that the new firmware has been installed.

    You can do this from the FIM GUI dashboard or from the FIM CLI using the get system status command.

  3. Verify that the configuration has been synchronized to the upgraded FIM. The following command output shows the synchronization status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command.If this does not solve the problem, contact Fortinet Support.

    The example output also shows that the uptime of the FIM in slot 2 is lower than the uptime of the other modules, indicating that the FIM in slot 2 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FIM has completely restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

Upgrading FPM firmware

Use the following procedure to upgrade the firmware running on an individual FPM. To perform the upgrade, you must enter a command from the primary FIM CLI to allow ELBC communication with the FPM. Then you can just log in to the FPM GUI or CLI and perform the firmware upgrade.

During this procedure, the FPM will not be able to process traffic. However, the other FPMs and the FIMs should continue to operate normally.

After verifying that the FPM is running the right firmware, you must log back into the primary FIM CLI and return the FPM to normal operation.

  1. Log in to the primary FIM CLI and enter the following command:

    diagnose load-balance switch set-compatible <slot> enable elbc

    Where <slot> is the number of the FortiGate-7000 slot containing the FPM to be upgraded.

  2. Log in to the FPM GUI or CLI using its special port number (for example, for the FPM in slot 3, browse to https://192.168.1.99:44303 to connect to the GUI) and perform a normal firmware upgrade of the FPM.

  3. After the FPM restarts, verify that the new firmware has been installed.

    You can do this from the FPM GUI dashboard or from the FPM CLI using the get system status command.

  4. Verify that the configuration has been synchronized. The following command output shows the sync status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command. If this does not solve the problem, contact Fortinet Support.

    The command output also shows that the uptime of the FPM in slot 4 is lower than the uptime of the other modules, indicating that the FPM in slot 4 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FIM has completely restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

  5. Once the FPM is operating normally, log back in to the primary FIM CLI and enter the following command to reset the FPM to normal operation:

    diagnose load-balance switch set-compatible <slot> disable

    Configuration synchronization errors will occur if you do not reset the FPM to normal operation.

Upgrading FIM firmware from the FIM BIOS using a TFTP server

Use the following procedure to upload firmware from a TFTP server to an FIM. The procedure involves creating a connection between the TFTP server and one of the FIM MGMT interfaces. You don't have to use a MGMT interface on the FIM that you are upgrading.

This procedure also involves connecting to the FIM CLI using a FortiGate-7000 front panel Management Module console port. From the console session, the procedure describes how to restart the FIM, interrupting the boot process, and follow FIM BIOS prompts to install the firmware.

During this procedure, the FIM will not be able to process traffic. However, the other FIM and the FPMs should continue to operate normally.

  1. Set up a TFTP server and copy the firmware file to the TFTP server default folder.

  2. Set up your network to allow traffic between the TFTP server and one of the FIM MGMT interfaces.

    If the MGMT interface you are using is one of the MGMT interfaces connected as a LAG to a switch, you must shutdown or disconnect all of the other connections in the LAG from the switch. This includes the MGMT interfaces in the other FIM.

  3. Using the console cable supplied with your FortiGate-7000, connect the management module Console 1 port on the FortiGate-7000 to the RS-232 port on your management computer.

  4. Start a terminal emulation program on the management computer. Use these settings:

    Baud Rate (bps) 9600, Data bits 8, Parity None, Stop bits 1, and Flow Control None.

  5. Press Ctrl-T to enter console switch mode.

  6. Repeat pressing Ctrl-T until you have connected to the FIM to be updated. Example prompt for the FIM in slot 2:

    <Switching to Console: FIM02 (9600)>

  7. Optionally log in to the FIM's CLI.

  8. Reboot the FIM.

    You can do this using the execute reboot command from the CLI or by pressing the power switch on the FIM front panel.

  9. When the FIM starts up, follow the boot process in the terminal session, and press any key when prompted to interrupt the boot process.

  10. To set up the TFTP configuration, press C.

  11. Use the BIOS menu to set the following. Change settings only if required.

    [P]: Set image download port: MGMT1 (the connected MGMT interface.)

    [D]: Set DHCP mode: Disabled

    [I]: Set local IP address: The IP address of the MGMT interface that you want to use to connect to the TFTP server. This address must not be the same as the FortiGate-7000 management IP address and cannot conflict with other addresses on your network.

    [S]: Set local Subnet Mask: Set as required for your network.

    [G]: Set local gateway: Set as required for your network.

    [V]: Local VLAN ID: Should be set to <none>. (use -1 to set the Local VLAN ID to <none>.)

    [T]: Set remote TFTP server IP address: The IP address of the TFTP server.

    [F]: Set firmware image file name: The name of the firmware image file that you want to install.

  12. To quit this menu, press Q.

  13. To review the configuration, press R.
    To make corrections, press C and make the changes as required. When the configuration is correct, proceed to the next step.

  14. To start the TFTP transfer, press T.

    The firmware image is uploaded from the TFTP server and installed on the FIM. The FIM then restarts with its configuration reset to factory defaults. After restarting, the FIM configuration is synchronized to match the configuration of the primary FIM. The FIM restarts again and can start processing traffic.

  15. Once the FIM restarts, verify that the correct firmware is installed.

    You can do this from the FIM GUI dashboard or from the FPM CLI using the get system status command.

  16. Verify that the configuration has been synchronized.

    The following command output shows the sync status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command. If this does not solve the problem, contact Fortinet support.

    The command output also shows that the uptime of the FIM in slot 2 is lower than the uptime of the other modules, indicating that the FIM in slot 2 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FIM has restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

Upgrading FPM firmware from the FPM BIOS using a TFTP server

Use the following procedure to upload firmware from a TFTP server to an FPM. To perform the upgrade, you must enter a command from the primary FIM CLI to allow the FPM BIOS to communicate through an FIM MGMT interface. The procedure involves creating a connection between the TFTP server and one of the FIM MGMT interfaces.

This procedure also involves connecting to the FPM CLI using a FortiGate-7000 front panel Management Module console port, rebooting the FPM, interrupting the boot from the console session, and following FPM BIOS prompts to install the firmware.

During this procedure, the FPM will not be able to process traffic. However, the other FPMs and the FIMs should continue to operate normally.

After you verify that the FPM is running the right firmware, you must log back in to the primary FIM CLI and return the FPM to normal operation.

  1. Set up a TFTP server and copy the firmware file into the TFTP server default folder.

  2. Log into to the primary FIM CLI and enter the following command:
    diagnose load-balance switch set-compatible <slot> enable bios

    Where <slot> is the number of the FortiGate-7000 slot containing the FPM to be upgraded.

  3. Set up your network to allow traffic between the TFTP server and a MGMT interface of one of the FIMs.

    You can use any MGMT interface of either of the FIMs. If the MGMT interface you are using is one of the MGMT interfaces connected as a LAG to a switch you must shutdown or disconnect all of the other connections in the LAG from the switch. This includes the MGMT interfaces in the other FIM.

  4. Using the console cable supplied with your FortiGate-7000, connect the management module Console 1 port on the FortiGate-7000 to the RS-232 port on your management computer.

  5. Start a terminal emulation program on the management computer. Use these settings:

    Baud Rate (bps) 9600, Data bits 8, Parity None, Stop bits 1, and Flow Control None.

  6. Press Ctrl-T to enter console switch mode.

  7. Repeat pressing Ctrl-T until you have connected to the module to be updated. Example prompt:
    <Switching to Console: FPM03 (9600)>

  8. Optionally log into the FPM's CLI.

  9. Reboot the FPM.

    You can do this using the execute reboot command from the FPM's CLI or by pressing the power switch on the FPM front panel.

  10. When the FPM starts up, follow the boot process in the terminal session and press any key when prompted to interrupt the boot process.

  11. To set up the TFTP configuration, press C.

  12. Use the BIOS menu to set the following. Change settings only if required.

    [P]: Set image download port: MGMT1 (the connected MGMT interface).

    [D]: Set DHCP mode: Disabled.

    [I]: Set local IP address: The IP address of the MGMT interface that you want to use to connect to the TFTP server. This address must not be the same as the FortiGate-7000 management IP address and cannot conflict with other addresses on your network.

    [S]: Set local Subnet Mask: Set as required for your network.

    [G]: Set local gateway: Set as required for your network.

    [V]: Local VLAN ID: Should be set to <none>. (use -1 to set the Local VLAN ID to <none>.)

    [T]: Set remote TFTP server IP address: The IP address of the TFTP server.

    [F]: Set firmware image file name: The name of the firmware image file that you want to install.

  13. To quit this menu, press Q.

  14. To review the configuration, press R.

    To make corrections, press C and make the changes as required. When the configuration is correct proceed to the next step.

  15. To start the TFTP transfer, press T.

    The firmware image is uploaded from the TFTP server and installed on the FPM. The FPM then restarts with its configuration reset to factory defaults. After restarting, the FPM configuration is synchronized to match the configuration of the primary FPM. The FPM restarts again and can start processing traffic.

  16. Once the FPM restarts, verify that the correct firmware is installed.

    You can do this from the FPM GUI dashboard or from the FPM CLI using the get system status command.

  17. Verify that the configuration has been synchronized.

    The following command output shows the sync status of a FortiGate-7040E. The field in_sync=1 indicates that the configurations of the FIMs and FPMs are synchronized.

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1

    FIMs and FPMs that are missing or that show in_sync=0 are not synchronized. To synchronize an FIM or FPM that is not synchronized, log into the CLI of the FIM or FPM and restart it using the execute reboot command . If this does not solve the problem, contact Fortinet Support.

    The command output also shows that the uptime of the FPM in slot 4 is lower than the uptime of the other modules, indicating that the FPM in slot 4 has recently restarted.

    If you enter the diagnose sys confsync status | grep in_sy command before the FPM has restarted, it will not appear in the command output. As well, the Security Fabric dashboard widget will temporarily show that it is not synchronized.

  18. Once the FPM is operating normally, log back in to the primary FIM CLI and enter the following command to reset the FPM to normal operation:
    diagnose load-balance switch set-compatible <slot> disable
    Configuration synchronization errors will occur if you do not reset the FPM to normal operation.

Synchronizing FIMs and FPMs after upgrading the primary FIM firmware from the BIOS

After you install firmware on the primary FIM from the BIOS after a reboot, the firmware version and configuration of the primary FIM will most likely be not be synchronized with the other FIMs and FPMs. You can verify this from the primary FIM CLI using the diagnose sys confsync status | grep in_sy command. The in_sync=0 entries in the following example output show that the management board (serial number ending in 10) is not synchronized with the other FIM and the FPMs shown in the example.

diagnose sys confsync status | grep in_sy
FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=0
FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=0
FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
...

You can also verify synchronization status from primary FIM Security Fabric dashboard widget.

To re-synchronize the FortiGate-7000, which has the effect of resetting the other FIM and the FPMs, re-install firmware on the primary FIM.

Note You can also manually install firmware on each individual FIM and FPM from the BIOS after a reboot. This manual process is just as effective as installing the firmware for a second time on the primary FIM to trigger synchronization to the FIM and the FPMs, but takes much longer.
  1. Log into the primary FIM GUI.

  2. Install a firmware build on the primary FIM from the GUI or CLI. The firmware build you install on the primary FIM can either be the same firmware build or a different one.

    Installing firmware synchronizes the firmware build and configuration from the primary FIM to the other FIM and the FPMs.

  3. Check the synchronization status from the Security Fabric dashboard widget or using the diagnose sys confsync status | grep in_sy command. The following example ForGate-7040E shows that the primary FIM is synchronized with the other FIM and all of the FPMs because each line includes in_sync=1:

    diagnose sys confsync status | grep in_sy
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x4, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FIM04E3E16000010, Master, uptime=69398.91, priority=1, slot_id=1:1, idx=0, flag=0x0, in_sync=1
    FIM10E3E16000040, Slave, uptime=69346.99, priority=2, slot_id=1:2, idx=1, flag=0x0, in_sync=1
    FPM20E3E17900217, Slave, uptime=69387.74, priority=20, slot_id=1:4, idx=2, flag=0x64, in_sync=1