Fortinet black logo

Upgrading FortiSOAR

Copy Link
Copy Doc ID 8e896ae4-4ead-11ed-9d74-fa163e15d75b:332067
Download PDF

Upgrading FortiSOAR

You must migrate the content from a pre-7.3.0 version to the 7.3.0 version as, there is a change in the supported OS. To migrate the content there are two methods:

Upgrading FortiSOAR using the FortiSOAR content export and restore method

This topic covers the upgrade process that requires you to export your database from the FortiSOAR 7.2.1 or 7.2.2 CentOS or RHEL system and import the into the newly setup Rocky Linux or RHEL system. For the FortiSOAR content export and restore method, use the migration script (migrate-fortisoar-7.3.0.bin).

Supported Migrations

  • FortiSOAR 7.2.1 or 7.2.2 CentOS 7.9 system to FortiSOAR 7.3.0 Rocky Linux or RHEL 8.6 system.
  • FortiSOAR 7.2.1 or 7.2.2 RHEL 7.0 system to FortiSOAR 7.3.0 Rocky Linux or RHEL 8.6 system.

Prerequisites

  • Setup a system with either Rocky Linux version 8.6 or RHEL version 8.6.
  • Users who have root access must run the migration script (migrate-fortisoar-7.3.0.bin), i.e., perform the FortiSOAR content export and restore operations.
  • Ensure that you have checked all the points mentioned in the Preparing to Upgrade FortiSOAR chapter.
  • Take a VM snapshot of your current system. Only after you have taken a VM snapshot of your system should you attempt to upgrade FortiSOAR. In case of any failures, these VM snapshots will allow you to revert to the latest working state. Follow the steps mentioned in the documentation of your platform for taking a snapshot and reverting to the current snapshot.
  • Review the list of failed notifications on FortiSOAR (Settings > Notifications > Failure Notifications Logs tab) before you upgrading your FortiSOAR instance.
  • Ensure that the ssh session does not timeout by entering into the screen mode. For more information on how to handle session timeouts, see the Handle session timeouts while running the FortiSOAR upgrade article present in the Fortinet Knowledge Base.

Migration Process

Exporting Data

  1. ssh to the FortiSOAR VM (CentOS 7.0 or RHEL 7.0) from which you want to export your data.
  2. Run the following command to download the migration script:
    # wget https://repo.fortisoar.fortinet.com/7.3.0/migrate-fortisoar-7.3.0.bin
    Note: If your instance can connect to "repo.fortisoar.fortinet.com" only by using a proxy, then ensure that the proxy is set in the /etc/wgetrc file. For example,
    use_proxy=yes
    http_proxy=<proxy_server_ip:port>
    https_proxy=<proxy_server_ip:port>
    You can also update the proxy setup using the csadm network command.
  3. Run the migrate script using the following command:
    # sh migrate-fortisoar-7.3.0.bin --export [<backup_dir_path>]
    OR
    # chmod +x migrate-fortisoar-7.3.0.bin]
    # ./migrate-fortisoar-7.3.0.bin --export [<backup_dir_path>
    [<backup_dir_path>] is the directory file to which the data [.tgz file] is exported. If you do not specify any path, then the migration script throws an appropriate error.
    Notes: The migration script checks for the following:
    • Free space available in the directory to which you are exporting the data. If there is insufficient space, then the script displays an appropriate error. You can refer to 'Issues occurring in FortiSOAR due to insufficient space' section in the Deployment Troubleshooting chapter in the "Deployment Guide" for more information. Once you increase the disk space rerun the migrate script to continue the process of exporting the data.
    • FortiSOAR version from which data is being exported. Export of data is supported only from FortiSOAR release 7.2.1 onwards.
    • OS from which data is being exported. Export of data is supported from only CentOS and RHEL.
  4. As a best practice move the database file to a different location as the VM from where the database file was exported is powered off.
  5. Once your data is exported, check the 'Summary' message, and note the following information:
    • Name of the file (.tgz) that contains the exported data.
    • Free disk space required for the database import process.
    • Disk space that must be available on the directory to which you want to import the data.

Importing Data

Before you begin importing the data, ensure that you do following:

  • Update the DNS as follows:
    1. Power off the system from where the data is exported.
    2. Update the DNS with the IP address of system where the data will be imported.
  • Disk space on the target system in which you want import the data is meets the requirements as mentioned in the 'Summary' of the export operation.

To import data to restore your database, do the following:

  1. ssh to the VM (RHEL or Rocky Linux) to which you want to import the data.
  2. Run the following command to copy the exported data (.tgz file) to the system on which you want to import the data:
    # scp <file_name.tgz> csadmin@targetipaddress:directory_path
    The targetipadress must be the IP address of the Rocky Linux or RHEL base OS.
    Note: The copy operation might take a longer time depending on the size of the database file.
    If the copy operation fails, then do the following:
    1. Check if the 'openssh-clients' is present on the target machine. If it is not present, then download and install openssh-clients using the following command:
      yum -y install openssh-clients
    2. Once openssh-clients is installed, re-run the scp command with the required parameters.
  3. Ensure that you are connected to a screen session.
  4. Run the following command to import the data to this system:
    # sh migrate-fortisoar-7.3.0.bin --import [<backup_file_path>]
    OR
    # chmod +x migrate-fortisoar-7.3.0.bin]
    # ./migrate-fortisoar-7.3.0.bin --import [<backup_file_path>
    [<backup_file_path>] is the name of the .tgz file that you want to import. If you do not specify the filename, then the migration script throws an appropriate error.
    Notes: The migration script checks for the following:
    • Free space available in the directory to which you are importing the data. If there is insufficient space, then the script displays an appropriate error. You can refer to 'Issues occurring in FortiSOAR due to insufficient space' section in the Deployment Troubleshooting chapter in the "Deployment Guide" for more information. Once you increase the disk space re-run the migrate script to continue the process of importing the data.
    • FortiSOAR version to which databases are being imported. Import of databases is supported only from FortiSOAR release 7.3.0 onwards.
    • OS to which data is being imported. Import of data is supported only to RHEL and Rocky Linux systems.
  5. The migration script imports the database and restores the hostname. Once the data is imported successfully, you can log into FortiSOAR and start using FortiSOAR.

Upgrading FortiSOAR using the in-place upgrade method

This topic covers the upgrade process using the in-place upgrade method. Upgrade of FortiSOAR using this method consists of the following phases:

  1. Preparing the FortiSOAR appliance
  2. Performing pre-checks on the FortiSOAR appliance
  3. Downloading and installing the required packages, and reconfiguring FortiSOAR

In-place upgrade paths

  • FortiSOAR 7.2.1 or 7.2.2 CentOS 7.9 system to FortiSOAR 7.3.0 Rocky Linux 8.6
  • FortiSOAR 7.2.1 or 7.2.2 RHEL 7.0 system to FortiSOAR 7.3.0 RHEL 8.6 system

In-place upgrade process

Preparing the system

To prepare the FortiSOAR appliance for running the in-place upgrade, do the following:

  1. Ensure that your FortiSOAR instance has the following minimum available space:
    • 2 GB free disk space in /opt
    • 2 GB free disk space in /var
  2. Ensure that repo.fortisoar.fortinet.com is reachable from your FortiSOAR appliance. OR, if you are in an air-gapped environment, then ensure that the offline repository is synced with release 7.2.1 or 7.2.2, whichever is the current release of FortiSOAR since, some of the upgrade packages are present on 7.2.1 and 7.2.2 repositories.
  3. In the case of an AWS instance, execute the following command to preserve /etc/resolv.conf:
    chattr +i /etc/resolv.conf
  4. Ensure that you temporarily unmount any NFS mounts before upgrading your system.
  5. If your system has more than one NIC, especially if the interface starts with eth, they must be temporarily disabled before the upgrad, i.e., ensure that there is only one NIC present before upgrading your system.

Performing pre-checks on the FortiSOAR appliance

This phase checks if your FortiSOAR appliance is ready for migration. If the pre-check displays warnings, the process stops, and the you are prompted to review the report and make changes, if necessary. No changes have been made to the system at this point.

  1. Connect to the FortiSOAR appliance terminal.
  2. Once ready, download the fortisoar-inplace-upgrade-7.3.0.bin utility, using wget or curl, to your FortiSOAR appliance from the following URL:
    https://repo.fortisoar.fortinet.com/7.3.0/fortisoar-inplace-upgrade-7.3.0.bin
  3. Enter into the screen mode. This allows to re-connect terminal session and keeps in-place upgrade process execution uninterrupted.
  4. Run the following command to launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin
    The migration utility gets launched and displays the following screen with informational message:
    In-place upgrade utility
  5. On the first prompt of the migration utility, type yes or y to continue with the pre-checks.
    If the pre-checks report some inhibitors for migration, the process stops, and a report file path is presented for review.
    You can make changes as per the report and re-execute the utility to continue with the pre-checks.
    Note: If the changes require time, then you can re-plan the migration activity to a time after the changes are made.
    At this point, no changes are done to the system.
  6. Once the pre-check process is clean, the migration utility asks for user confirmation to continue the migration process.

Downloading and installing the required packages, and reconfiguring FortiSOAR

  1. Once the user confirmation is received for the continuation of the migration process once the pre-check process is clean, the migration utility performs the following steps:
    1. Cleans up unsupported drivers and other packages.
    2. Downloads FortiSOAR and OS packages.
    3. Once all packages are downloaded, the migration utility displays a message asking the user to reboot the system. A manual system reboot is required.
      During system reboot, FortiSOAR and the required packages are installed and due to this terminal access will be blocked for 15-20 min.
  2. Once the system gets rebooted and the terminal access is restored , the migration utility continues with the FortiSOAR migration.
    You can monitor the migration process by checking /var/log/leapp/leapp-upgrade.log.
    Once the migration process is completed successfully the utility displays a completion message.
    Note: This process takes approximately one hour to complete.

Manual steps for reconfiguring FortiSOAR after the OS Upgrade when the 'leapp' service fails to resume in Enterprise or MSSP configurations

As explained, the in-place upgrade has two phases: a pre-OS upgrade and a post-OS upgrade. During the pre-OS upgrade phase, a FortiSOAR backup is taken and the system is prepared for the OS upgrade. The system requires to be rebooted after the pre-OS upgrade phase. Once the system is rebooted, the leapp framework attempts to initiate the second phase. In the second phase, FortiSOAR is reconfigured with the latest version and the backed up configurations are restored as mentioned in the Downloading and installing the required packages, and reconfiguring FortiSOAR topic. In the rare cases where the leapp service does not work or fails to resume, you can manually reconfigure FortiSOAR with the latest version and restore the backed-up configurations, as described in the following topics.

Pre-Checks
  1. Check /var/log/leapp/leapp-upgrade.log:
    1. Search for the term 'fortisoarupgrade' to see if there was any error.
    2. If the error has occurred post installation of FortiSOAR, consult Support. If FortiSOAR was not installed, then proceed with the steps.
  2. Check for the existence of the backup file. The filename of the backup file starts with DR_XYS.tar
  3. After the OS is rebooted, check if FortiSOAR version 7.3.0 is installed.
  4. Check for the existence of the /var/lib/pgsql/12_bkp directory.
  5. Install tmux.
  6. If your system uses a proxy, update the proxy configuration for yum and in the /etc/environment file.
Manual Steps
  1. Launch tmux.
  2. For performing the manual steps on a FortiSOAR MSSP or Enterprise setup, set the following environmental variable using the following command:
    export fsr_edition=enterprise
  3. Set the following OS environment variable using the following command:
    export is_inplace_upgrade=true
  4. If FortiSOAR version 7.3.0 is not installed, then download and install FortiSOAR 7.3.0. If FortiSOAR is already installed, then skip this step.
    1. wget https://repo.fortisoar.fortinet.com/7.3.0/install-fortisoar-7.3.0.bin
    2. Start the installation by executing the following command:
      sh install-fortisoar-7.3.0.bin
    3. Follow the on-screen instructions and complete the installation.
  5. Rename the database backup directory:
    mv /var/lib/pgsql/12_bkp /var/lib/pgsql/12
  6. Once FortiSOAR is installed, begin reconfiguring FortiSOAR:
    1. Download the migration script using the following command:
      # wget https://repo.fortisoar.fortinet.com/7.3.0/migrate-fortisoar-7.3.0.bin
    2. Initiate the restoration of your FortiSOAR configurations using the following command:
      # sh migrate-fortisoar-7.3.0.bin --import [<backup_dir_path>]
    3. Follow the on-screen instructions and complete the restoration of your FortiSOAR configurations.
  7. Once your FortiSOAR configurations are restored, run the following command:
    sed -i 's/trust/md5/g' /var/lib/pgsql/14/data/pg_hba.conf
  8. Install 'Ansible' using the following command:
    /bin/sudo -u nginx /opt/cyops-workflow/.env/bin/pip install ansible==2.8.2 --extra-index-url https://repo.fortisoar.fortinet.com/prod/connectors/deps/simple/
  9. Restart the FortiSOAR services using the following command:
    csadm services –restart
  10. Update the login banner as follows:
    Edit the /etc/motd file to change the 'CentOS Version: 7.9.2009' string to 'Rocky Linux Version: 8.6'
  11. Update the attributes in the resolv.conf file using the following command:
    /bin/chattr -i filepath
    This completes the FortiSOAR upgrade to release 7.3.0.

Upgrading FortiSOAR with an externalized database

  1. Use in place upgrade script to upgrade to 7.3.0, see the Upgrading FortiSOAR using the in-place upgrade method topic.
  2. Once FortiSOAR is successfully upgraded, upgrade your PostGreSQL version to 14, if you are using self-deployed PostGreSQL.
    If you are using Cloud, then use the cloud service to migrate to PostGreSQL14

Upgrade considerations for varied types of FortiSOAR configurations

You can upgrade the following FortiSOAR configuration:

  • FortiSOAR Enterprise Edition - Standalone and HA.
    Notes:
    All nodes within an HA cluster must be upgraded to the same release of FortiSOAR.
    Post-upgrade, you will need to reconfigure the HA cluster.
    For more information, see the Upgrading a FortiSOAR High Availability Cluster chapter.
  • FortiSOAR MSSP Edition - Standalone and HA
    Notes:
    In the case of an MSSP setup, the master, tenant, and secure exchange message nodes must be upgraded to the same release of FortiSOAR.
    For more information, see the Upgrading a FortiSOAR Distributed Multi-Tenancy Configuration chapter.
  • FortiSOAR on FortiCloud. For more information, see the Upgrade Information chapter in the "FortiSOAR Cloud 7.3.0 Release Notes".

Upgrade Notes

  • The postfix version bundled with FortiSOAR 7.3.0 has multiple "Safety Net" improvements. So that customer workflows built with the SMTP connector and using the local postfix configuration are not blocked, the postfix configuration is updated to the compatibility_level of '2'. However, it is strongly recommended that you apply the new settings such as smtpd_relay_restriction to have good relay restrictions, etc. For more information, see the Postfix Backwards-Compatibility Safety Net article.

Troubleshooting in-place upgrade failures

The in-place upgrade can fail for various reasons, and you can resolve them by checking the reason of the failure by reviewing the leapp-report.txt file located at /var/log/leapp/leapp-report.txt. The leapp-report.txt details the risk factor's level of severity, the title and summary of the failure reason, and the issue's solution. You can restart the upgrade after fixing the problem as advised in the leapp-report.txt file. A copy of the leapp-report.txt file is also saved in location from where the upgrade script is executed.

In-place upgrades fail on setups that have their proxy configured

If you have a system that has its proxy configured, then the in-place upgrade fails with the following error:
Failed to synchronize cache for repo

Resolution

On your appliance terminal, run the following command:
export LEAPP_PROXY_HOST=http://<ADDRESS>:<PORT>
Replace <address> and <port> with your actual proxy.

If this solution does not work, then do the following:

  1. Run the following command:
    sed -i '/^enabled=.*/a proxy=<proxy server>' /etc/leapp/files/leapp_upgrade_repositories.repo
    Replace <proxy server> with the actual proxy server.
  2. Run the following command:
    execute truncate -s -1 fortisoar-inplace-upgrade-7.3.0.bin
  3. Restart the upgrade procedure by running the following command to re-launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin

In-place upgrades fail with the 'Possible problems with remote login using root account' error

This issue occurs when you have deployed a Centos VM where root is allowed to login. However, as per RHEL8 policy root is not allowed for "ssh" access and requires to blocked, leading to pre-upgrade checks failing with the Possible problems with remote login using root account error.

Resolution

  1. Add a "non-root" user with 'admin' privileges and disable ssh login for the root user.
    For additional information, you can also check the leapp-report.txt file.
  2. Restart the upgrade procedure by running the following command to re-launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin

Pre-upgrade check for in-place upgrade fails with 'kernel' issues

The the pre-upgrade check fails with the following errors when you have updated a system that causes the kernel to upgrade, but you have forgotten to reboot the system after the upgrade:

###################
Upgrade has been inhibited due to the following problems:

    1. Inhibitor: Multiple devel kernels installed

    2. Inhibitor: Newest installed kernel not in use

Consult the pre-upgrade report for details and possible remediation.
######################

Resolution

  1. Reboot your system.
  2. Restart the upgrade procedure by running the following command to re-launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin

Pre-upgrade check for in-place upgrade fails with a 'synchronization error for the fsr repositories' error

The pre-upgrade checks fail with a synchronization error for the fsr repositories, for example,
"Failed to synchronize cache for repo 'fsr-rockylinux-baseos', ignoring this repo."

This issue occurs when yum fails to recognize the proxy settings and therefore, the proxy settings need to be explicitly configured for each repository.

Resolution

  1. Manually set the proxy for each repository using the following command:
    sed -i '/^enabled=.*/a proxy=<proxy>' fortisoar-inplace-upgrade-7.3.0.bin
    For example, sed -i '/^enabled=.*/a proxy=http://user:password@10.10.10.10:808' fortisoar-inplace-upgrade-7.3.0.bin
  2. Refresh the binary to apply the proxy changes using the following command:
    truncate -s -1 fortisoar-inplace-upgrade-7.3.0.bin
  3. Re-run the upgrade script using the following command:
    sh fortisoar-inplace-upgrade-7.3.0.bin

Troubleshooting upgrade issues

This topic provides you with troubleshooting tips, irrespective of the selected upgrade method.

Retrying of failed notifications might fail

Before you start upgrading FortiSOAR, you must review failed notifications. After the upgrade, you can retry a failed notification by clicking Settings > Notifications > Failure Notifications Logs tab, selecting the failed notification, and then clicking Retry Notification. This "Retry" process might still fail.

Resolution

If the "Retry" for failed notifications still fails after the upgrade, then restart the fsr-workflow (systemctl restart celeryd celerybeatd fsr-workflow) and uwsgi (# systemctl restart uwsgi) services.

Error messages displayed during the restore process

During the post-reboot phase in the case of an in-place upgrade or the database restore phase in the case of an upgrade using the export-import method errors such as the following is displayed:
Error: unable to perform an operation on node 'rabbit@qa-sme-720-sys1'. Please see diagnostics information and suggestions below.

Resolution

You can ignore these errors because they show that your configurations are currently being restored.

Upgrading FortiSOAR

You must migrate the content from a pre-7.3.0 version to the 7.3.0 version as, there is a change in the supported OS. To migrate the content there are two methods:

Upgrading FortiSOAR using the FortiSOAR content export and restore method

This topic covers the upgrade process that requires you to export your database from the FortiSOAR 7.2.1 or 7.2.2 CentOS or RHEL system and import the into the newly setup Rocky Linux or RHEL system. For the FortiSOAR content export and restore method, use the migration script (migrate-fortisoar-7.3.0.bin).

Supported Migrations

  • FortiSOAR 7.2.1 or 7.2.2 CentOS 7.9 system to FortiSOAR 7.3.0 Rocky Linux or RHEL 8.6 system.
  • FortiSOAR 7.2.1 or 7.2.2 RHEL 7.0 system to FortiSOAR 7.3.0 Rocky Linux or RHEL 8.6 system.

Prerequisites

  • Setup a system with either Rocky Linux version 8.6 or RHEL version 8.6.
  • Users who have root access must run the migration script (migrate-fortisoar-7.3.0.bin), i.e., perform the FortiSOAR content export and restore operations.
  • Ensure that you have checked all the points mentioned in the Preparing to Upgrade FortiSOAR chapter.
  • Take a VM snapshot of your current system. Only after you have taken a VM snapshot of your system should you attempt to upgrade FortiSOAR. In case of any failures, these VM snapshots will allow you to revert to the latest working state. Follow the steps mentioned in the documentation of your platform for taking a snapshot and reverting to the current snapshot.
  • Review the list of failed notifications on FortiSOAR (Settings > Notifications > Failure Notifications Logs tab) before you upgrading your FortiSOAR instance.
  • Ensure that the ssh session does not timeout by entering into the screen mode. For more information on how to handle session timeouts, see the Handle session timeouts while running the FortiSOAR upgrade article present in the Fortinet Knowledge Base.

Migration Process

Exporting Data

  1. ssh to the FortiSOAR VM (CentOS 7.0 or RHEL 7.0) from which you want to export your data.
  2. Run the following command to download the migration script:
    # wget https://repo.fortisoar.fortinet.com/7.3.0/migrate-fortisoar-7.3.0.bin
    Note: If your instance can connect to "repo.fortisoar.fortinet.com" only by using a proxy, then ensure that the proxy is set in the /etc/wgetrc file. For example,
    use_proxy=yes
    http_proxy=<proxy_server_ip:port>
    https_proxy=<proxy_server_ip:port>
    You can also update the proxy setup using the csadm network command.
  3. Run the migrate script using the following command:
    # sh migrate-fortisoar-7.3.0.bin --export [<backup_dir_path>]
    OR
    # chmod +x migrate-fortisoar-7.3.0.bin]
    # ./migrate-fortisoar-7.3.0.bin --export [<backup_dir_path>
    [<backup_dir_path>] is the directory file to which the data [.tgz file] is exported. If you do not specify any path, then the migration script throws an appropriate error.
    Notes: The migration script checks for the following:
    • Free space available in the directory to which you are exporting the data. If there is insufficient space, then the script displays an appropriate error. You can refer to 'Issues occurring in FortiSOAR due to insufficient space' section in the Deployment Troubleshooting chapter in the "Deployment Guide" for more information. Once you increase the disk space rerun the migrate script to continue the process of exporting the data.
    • FortiSOAR version from which data is being exported. Export of data is supported only from FortiSOAR release 7.2.1 onwards.
    • OS from which data is being exported. Export of data is supported from only CentOS and RHEL.
  4. As a best practice move the database file to a different location as the VM from where the database file was exported is powered off.
  5. Once your data is exported, check the 'Summary' message, and note the following information:
    • Name of the file (.tgz) that contains the exported data.
    • Free disk space required for the database import process.
    • Disk space that must be available on the directory to which you want to import the data.

Importing Data

Before you begin importing the data, ensure that you do following:

  • Update the DNS as follows:
    1. Power off the system from where the data is exported.
    2. Update the DNS with the IP address of system where the data will be imported.
  • Disk space on the target system in which you want import the data is meets the requirements as mentioned in the 'Summary' of the export operation.

To import data to restore your database, do the following:

  1. ssh to the VM (RHEL or Rocky Linux) to which you want to import the data.
  2. Run the following command to copy the exported data (.tgz file) to the system on which you want to import the data:
    # scp <file_name.tgz> csadmin@targetipaddress:directory_path
    The targetipadress must be the IP address of the Rocky Linux or RHEL base OS.
    Note: The copy operation might take a longer time depending on the size of the database file.
    If the copy operation fails, then do the following:
    1. Check if the 'openssh-clients' is present on the target machine. If it is not present, then download and install openssh-clients using the following command:
      yum -y install openssh-clients
    2. Once openssh-clients is installed, re-run the scp command with the required parameters.
  3. Ensure that you are connected to a screen session.
  4. Run the following command to import the data to this system:
    # sh migrate-fortisoar-7.3.0.bin --import [<backup_file_path>]
    OR
    # chmod +x migrate-fortisoar-7.3.0.bin]
    # ./migrate-fortisoar-7.3.0.bin --import [<backup_file_path>
    [<backup_file_path>] is the name of the .tgz file that you want to import. If you do not specify the filename, then the migration script throws an appropriate error.
    Notes: The migration script checks for the following:
    • Free space available in the directory to which you are importing the data. If there is insufficient space, then the script displays an appropriate error. You can refer to 'Issues occurring in FortiSOAR due to insufficient space' section in the Deployment Troubleshooting chapter in the "Deployment Guide" for more information. Once you increase the disk space re-run the migrate script to continue the process of importing the data.
    • FortiSOAR version to which databases are being imported. Import of databases is supported only from FortiSOAR release 7.3.0 onwards.
    • OS to which data is being imported. Import of data is supported only to RHEL and Rocky Linux systems.
  5. The migration script imports the database and restores the hostname. Once the data is imported successfully, you can log into FortiSOAR and start using FortiSOAR.

Upgrading FortiSOAR using the in-place upgrade method

This topic covers the upgrade process using the in-place upgrade method. Upgrade of FortiSOAR using this method consists of the following phases:

  1. Preparing the FortiSOAR appliance
  2. Performing pre-checks on the FortiSOAR appliance
  3. Downloading and installing the required packages, and reconfiguring FortiSOAR

In-place upgrade paths

  • FortiSOAR 7.2.1 or 7.2.2 CentOS 7.9 system to FortiSOAR 7.3.0 Rocky Linux 8.6
  • FortiSOAR 7.2.1 or 7.2.2 RHEL 7.0 system to FortiSOAR 7.3.0 RHEL 8.6 system

In-place upgrade process

Preparing the system

To prepare the FortiSOAR appliance for running the in-place upgrade, do the following:

  1. Ensure that your FortiSOAR instance has the following minimum available space:
    • 2 GB free disk space in /opt
    • 2 GB free disk space in /var
  2. Ensure that repo.fortisoar.fortinet.com is reachable from your FortiSOAR appliance. OR, if you are in an air-gapped environment, then ensure that the offline repository is synced with release 7.2.1 or 7.2.2, whichever is the current release of FortiSOAR since, some of the upgrade packages are present on 7.2.1 and 7.2.2 repositories.
  3. In the case of an AWS instance, execute the following command to preserve /etc/resolv.conf:
    chattr +i /etc/resolv.conf
  4. Ensure that you temporarily unmount any NFS mounts before upgrading your system.
  5. If your system has more than one NIC, especially if the interface starts with eth, they must be temporarily disabled before the upgrad, i.e., ensure that there is only one NIC present before upgrading your system.

Performing pre-checks on the FortiSOAR appliance

This phase checks if your FortiSOAR appliance is ready for migration. If the pre-check displays warnings, the process stops, and the you are prompted to review the report and make changes, if necessary. No changes have been made to the system at this point.

  1. Connect to the FortiSOAR appliance terminal.
  2. Once ready, download the fortisoar-inplace-upgrade-7.3.0.bin utility, using wget or curl, to your FortiSOAR appliance from the following URL:
    https://repo.fortisoar.fortinet.com/7.3.0/fortisoar-inplace-upgrade-7.3.0.bin
  3. Enter into the screen mode. This allows to re-connect terminal session and keeps in-place upgrade process execution uninterrupted.
  4. Run the following command to launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin
    The migration utility gets launched and displays the following screen with informational message:
    In-place upgrade utility
  5. On the first prompt of the migration utility, type yes or y to continue with the pre-checks.
    If the pre-checks report some inhibitors for migration, the process stops, and a report file path is presented for review.
    You can make changes as per the report and re-execute the utility to continue with the pre-checks.
    Note: If the changes require time, then you can re-plan the migration activity to a time after the changes are made.
    At this point, no changes are done to the system.
  6. Once the pre-check process is clean, the migration utility asks for user confirmation to continue the migration process.

Downloading and installing the required packages, and reconfiguring FortiSOAR

  1. Once the user confirmation is received for the continuation of the migration process once the pre-check process is clean, the migration utility performs the following steps:
    1. Cleans up unsupported drivers and other packages.
    2. Downloads FortiSOAR and OS packages.
    3. Once all packages are downloaded, the migration utility displays a message asking the user to reboot the system. A manual system reboot is required.
      During system reboot, FortiSOAR and the required packages are installed and due to this terminal access will be blocked for 15-20 min.
  2. Once the system gets rebooted and the terminal access is restored , the migration utility continues with the FortiSOAR migration.
    You can monitor the migration process by checking /var/log/leapp/leapp-upgrade.log.
    Once the migration process is completed successfully the utility displays a completion message.
    Note: This process takes approximately one hour to complete.

Manual steps for reconfiguring FortiSOAR after the OS Upgrade when the 'leapp' service fails to resume in Enterprise or MSSP configurations

As explained, the in-place upgrade has two phases: a pre-OS upgrade and a post-OS upgrade. During the pre-OS upgrade phase, a FortiSOAR backup is taken and the system is prepared for the OS upgrade. The system requires to be rebooted after the pre-OS upgrade phase. Once the system is rebooted, the leapp framework attempts to initiate the second phase. In the second phase, FortiSOAR is reconfigured with the latest version and the backed up configurations are restored as mentioned in the Downloading and installing the required packages, and reconfiguring FortiSOAR topic. In the rare cases where the leapp service does not work or fails to resume, you can manually reconfigure FortiSOAR with the latest version and restore the backed-up configurations, as described in the following topics.

Pre-Checks
  1. Check /var/log/leapp/leapp-upgrade.log:
    1. Search for the term 'fortisoarupgrade' to see if there was any error.
    2. If the error has occurred post installation of FortiSOAR, consult Support. If FortiSOAR was not installed, then proceed with the steps.
  2. Check for the existence of the backup file. The filename of the backup file starts with DR_XYS.tar
  3. After the OS is rebooted, check if FortiSOAR version 7.3.0 is installed.
  4. Check for the existence of the /var/lib/pgsql/12_bkp directory.
  5. Install tmux.
  6. If your system uses a proxy, update the proxy configuration for yum and in the /etc/environment file.
Manual Steps
  1. Launch tmux.
  2. For performing the manual steps on a FortiSOAR MSSP or Enterprise setup, set the following environmental variable using the following command:
    export fsr_edition=enterprise
  3. Set the following OS environment variable using the following command:
    export is_inplace_upgrade=true
  4. If FortiSOAR version 7.3.0 is not installed, then download and install FortiSOAR 7.3.0. If FortiSOAR is already installed, then skip this step.
    1. wget https://repo.fortisoar.fortinet.com/7.3.0/install-fortisoar-7.3.0.bin
    2. Start the installation by executing the following command:
      sh install-fortisoar-7.3.0.bin
    3. Follow the on-screen instructions and complete the installation.
  5. Rename the database backup directory:
    mv /var/lib/pgsql/12_bkp /var/lib/pgsql/12
  6. Once FortiSOAR is installed, begin reconfiguring FortiSOAR:
    1. Download the migration script using the following command:
      # wget https://repo.fortisoar.fortinet.com/7.3.0/migrate-fortisoar-7.3.0.bin
    2. Initiate the restoration of your FortiSOAR configurations using the following command:
      # sh migrate-fortisoar-7.3.0.bin --import [<backup_dir_path>]
    3. Follow the on-screen instructions and complete the restoration of your FortiSOAR configurations.
  7. Once your FortiSOAR configurations are restored, run the following command:
    sed -i 's/trust/md5/g' /var/lib/pgsql/14/data/pg_hba.conf
  8. Install 'Ansible' using the following command:
    /bin/sudo -u nginx /opt/cyops-workflow/.env/bin/pip install ansible==2.8.2 --extra-index-url https://repo.fortisoar.fortinet.com/prod/connectors/deps/simple/
  9. Restart the FortiSOAR services using the following command:
    csadm services –restart
  10. Update the login banner as follows:
    Edit the /etc/motd file to change the 'CentOS Version: 7.9.2009' string to 'Rocky Linux Version: 8.6'
  11. Update the attributes in the resolv.conf file using the following command:
    /bin/chattr -i filepath
    This completes the FortiSOAR upgrade to release 7.3.0.

Upgrading FortiSOAR with an externalized database

  1. Use in place upgrade script to upgrade to 7.3.0, see the Upgrading FortiSOAR using the in-place upgrade method topic.
  2. Once FortiSOAR is successfully upgraded, upgrade your PostGreSQL version to 14, if you are using self-deployed PostGreSQL.
    If you are using Cloud, then use the cloud service to migrate to PostGreSQL14

Upgrade considerations for varied types of FortiSOAR configurations

You can upgrade the following FortiSOAR configuration:

  • FortiSOAR Enterprise Edition - Standalone and HA.
    Notes:
    All nodes within an HA cluster must be upgraded to the same release of FortiSOAR.
    Post-upgrade, you will need to reconfigure the HA cluster.
    For more information, see the Upgrading a FortiSOAR High Availability Cluster chapter.
  • FortiSOAR MSSP Edition - Standalone and HA
    Notes:
    In the case of an MSSP setup, the master, tenant, and secure exchange message nodes must be upgraded to the same release of FortiSOAR.
    For more information, see the Upgrading a FortiSOAR Distributed Multi-Tenancy Configuration chapter.
  • FortiSOAR on FortiCloud. For more information, see the Upgrade Information chapter in the "FortiSOAR Cloud 7.3.0 Release Notes".

Upgrade Notes

  • The postfix version bundled with FortiSOAR 7.3.0 has multiple "Safety Net" improvements. So that customer workflows built with the SMTP connector and using the local postfix configuration are not blocked, the postfix configuration is updated to the compatibility_level of '2'. However, it is strongly recommended that you apply the new settings such as smtpd_relay_restriction to have good relay restrictions, etc. For more information, see the Postfix Backwards-Compatibility Safety Net article.

Troubleshooting in-place upgrade failures

The in-place upgrade can fail for various reasons, and you can resolve them by checking the reason of the failure by reviewing the leapp-report.txt file located at /var/log/leapp/leapp-report.txt. The leapp-report.txt details the risk factor's level of severity, the title and summary of the failure reason, and the issue's solution. You can restart the upgrade after fixing the problem as advised in the leapp-report.txt file. A copy of the leapp-report.txt file is also saved in location from where the upgrade script is executed.

In-place upgrades fail on setups that have their proxy configured

If you have a system that has its proxy configured, then the in-place upgrade fails with the following error:
Failed to synchronize cache for repo

Resolution

On your appliance terminal, run the following command:
export LEAPP_PROXY_HOST=http://<ADDRESS>:<PORT>
Replace <address> and <port> with your actual proxy.

If this solution does not work, then do the following:

  1. Run the following command:
    sed -i '/^enabled=.*/a proxy=<proxy server>' /etc/leapp/files/leapp_upgrade_repositories.repo
    Replace <proxy server> with the actual proxy server.
  2. Run the following command:
    execute truncate -s -1 fortisoar-inplace-upgrade-7.3.0.bin
  3. Restart the upgrade procedure by running the following command to re-launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin

In-place upgrades fail with the 'Possible problems with remote login using root account' error

This issue occurs when you have deployed a Centos VM where root is allowed to login. However, as per RHEL8 policy root is not allowed for "ssh" access and requires to blocked, leading to pre-upgrade checks failing with the Possible problems with remote login using root account error.

Resolution

  1. Add a "non-root" user with 'admin' privileges and disable ssh login for the root user.
    For additional information, you can also check the leapp-report.txt file.
  2. Restart the upgrade procedure by running the following command to re-launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin

Pre-upgrade check for in-place upgrade fails with 'kernel' issues

The the pre-upgrade check fails with the following errors when you have updated a system that causes the kernel to upgrade, but you have forgotten to reboot the system after the upgrade:

###################
Upgrade has been inhibited due to the following problems:

    1. Inhibitor: Multiple devel kernels installed

    2. Inhibitor: Newest installed kernel not in use

Consult the pre-upgrade report for details and possible remediation.
######################

Resolution

  1. Reboot your system.
  2. Restart the upgrade procedure by running the following command to re-launch the migration tool:
    sh fortisoar-inplace-upgrade-7.3.0.bin

Pre-upgrade check for in-place upgrade fails with a 'synchronization error for the fsr repositories' error

The pre-upgrade checks fail with a synchronization error for the fsr repositories, for example,
"Failed to synchronize cache for repo 'fsr-rockylinux-baseos', ignoring this repo."

This issue occurs when yum fails to recognize the proxy settings and therefore, the proxy settings need to be explicitly configured for each repository.

Resolution

  1. Manually set the proxy for each repository using the following command:
    sed -i '/^enabled=.*/a proxy=<proxy>' fortisoar-inplace-upgrade-7.3.0.bin
    For example, sed -i '/^enabled=.*/a proxy=http://user:password@10.10.10.10:808' fortisoar-inplace-upgrade-7.3.0.bin
  2. Refresh the binary to apply the proxy changes using the following command:
    truncate -s -1 fortisoar-inplace-upgrade-7.3.0.bin
  3. Re-run the upgrade script using the following command:
    sh fortisoar-inplace-upgrade-7.3.0.bin

Troubleshooting upgrade issues

This topic provides you with troubleshooting tips, irrespective of the selected upgrade method.

Retrying of failed notifications might fail

Before you start upgrading FortiSOAR, you must review failed notifications. After the upgrade, you can retry a failed notification by clicking Settings > Notifications > Failure Notifications Logs tab, selecting the failed notification, and then clicking Retry Notification. This "Retry" process might still fail.

Resolution

If the "Retry" for failed notifications still fails after the upgrade, then restart the fsr-workflow (systemctl restart celeryd celerybeatd fsr-workflow) and uwsgi (# systemctl restart uwsgi) services.

Error messages displayed during the restore process

During the post-reboot phase in the case of an in-place upgrade or the database restore phase in the case of an upgrade using the export-import method errors such as the following is displayed:
Error: unable to perform an operation on node 'rabbit@qa-sme-720-sys1'. Please see diagnostics information and suggestions below.

Resolution

You can ignore these errors because they show that your configurations are currently being restored.