Fortinet black logo

Upgrading a FortiSOAR Distributed Multi-Tenancy Configuration

Copy Link
Copy Doc ID 0b62b19d-1178-11ed-9eba-fa163e15d75b:28640
Download PDF

Upgrading a FortiSOAR Distributed Multi-Tenancy Configuration

This section describes the procedure to upgrade a FortiSOAR distributed multi-tenant configuration for managed security services providers (MSSPs) or Distributed SOC configuration.

You must first upgrade the master node of your FortiSOAR distributed multi-tenant configuration and only then upgrade the tenant nodes of your FortiSOAR multi-tenancy setup.

Caution

In case of a distributed deployment, both the master and the tenant nodes must be upgraded. A version mismatch will not work if either of them upgrades to 7.2.2.

Upgrading a FortiSOAR master node

Before you upgrade your FortiSOAR master node, ensure the following:

  • All playbooks have completed their execution on the master.
  • The tenant node(s) are deactivated from the master node before upgrading the master node.

If the master node of your multi-tenant configuration is part of an HA setup, i.e., MSSP +HA, then follow the steps mentioned in the Upgrading a FortiSOAR High Availability Cluster section.

If the master node of your multi-tenant configuration is not part of an HA setup, then follow the steps mentioned in the Upgrading a FortiSOAR Enterprise Instance section.

Upgrading a FortiSOAR Tenant node

Before you upgrade your FortiSOAR tenant node, ensure the following:

  • Data replication from the tenant node to the master node is stopped. You can stop data replication by logging on to the tenant node and clicking Settings to open the System page, then in the Multi Tenancy section, click the Master Configuration menu item and then in the Communication With Master Node section, toggle the Enabled button to NO.
  • All playbooks have completed their execution on the tenant.
  • All schedule playbooks that fetch data from data sources to the tenant are stopped.
  • Any application that pushes data from data sources to the tenant is stopped.

If the tenant node of your multi-tenant configuration is part of an HA setup, i.e., MSSP +HA, then follow the steps mentioned in the Upgrading a FortiSOAR High Availability Cluster section.

If the tenant node of your multi-tenant configuration is not part of an HA setup, then follow the steps mentioned in the Upgrading a FortiSOAR Enterprise Instance section.

Note After the tenant node has been successfully upgraded, you must toggle the Allow Module Management setting to NO and then back to YES. This is needed only if you were already using the 'Allow Module Management' feature and is required to synchronize the tenant module metadata with the master instance. You can ignore this step, if your 'Allow Module Management' setting was already disabled before the upgrade.

Upgrading a FortiSOAR Secure Message Exchange

A secure message exchange establishes a secure channel that is used to relay information to the agents or tenant nodes. To create a dedicated secure channel, you are required to add the reference of the installed and configured secure message exchange, when you add agent or tenant nodes to your environment. For information on agents see the Segmented Network support in FortiSOAR chapter in the "Administration Guide," and for more information on secure message exchange and tenants, see the "Multi-Tenancy support in FortiSOAR Guide".

  1. Ensure that you stop data replication between the master and the tenant nodes. You can stop data replication by logging on to the tenant node and clicking Settings to open the System page, then in the Multi Tenancy section, click the Master Configuration menu item and then in the Communication With Master Node section, toggle the Enabled button to NO.
  2. SSH to the secure message exchange VM that you want to upgrade.
  3. Check that you are connected to a screen session. A screen session is needed for situations where network connectivity is less than favorable. You can check your screen session using the following command:
    # screen -ls
    This command returns an output such as the following example:
    There is a screen on:
    12081.upgrade(Detached)

    Log back into the SSH console and run the following command to reattach the screen session:
    screen -r 12081.upgrade
    OR
    screen -r upgrade
  4. Run the following command to download the upgrade installer:
    # wget https://repo.fortisoar.fortinet.com/7.2.2/upgrade-fortisoar-7.2.2.bin
  5. Run the upgrade installer using the following command:
    # sh upgrade-fortisoar-7.2.2.bin
    OR
    # chmod +x upgrade-fortisoar-7.2.2.bin
    # ./upgrade-fortisoar-7.2.2.bin
    Notes: The FortiSOAR upgrade installer checks for the following:
    • The space available in /tmp or /var/temp (if exist in /etc/fstab), which must be at least 2GB. If the space available in in /tmp or /var/temp is less than 2GB, then the upgrade installer exits after displaying an appropriate error message. If you want to skip this space validation check, you can use the --skip-tmp-validation option while running the upgrade script:
      # ./upgrade-fortisoar-7.2.2.bin --skip-tmp-validation
    • The disk space available in /boot, and if the /boot has insufficient space, then the upgrade installer exits after displaying an appropriate error message. Steps for cleaning up /boot are present in the Clean up /boot article present in the Fortinet Knowledge Base.
    • The disk space available in /var/lib/pgsql to ensure that you have sufficient disk space for pgsql. If you do not have sufficient disk space for pgsql, in this case also the upgrade installer exits. In these cases, you must increase the partition size for /var/lib/pgsql. For the procedure to increase the partition size, see the 'Issues occurring in FortiSOAR due to insufficient space' section in the Deployment Troubleshooting chapter in the "Deployment Guide" for more information.
    Once you complete cleaning up /boot and/or increasing disk space and space in /tmp (as per the messages provided by the upgrade installer) and you have sufficient space for upgrading FortiSOAR, you must re-run the upgrade installer to continue the process of upgrading FortiSOAR.
  6. Once you have successfully upgraded the secure message exchange, start the data replication between the master and the tenant nodes again by toggling the Data Replication button to ON, and then verify the replication.

Upgrading a FortiSOAR Secure Message Exchange Cluster

RabbitMQ supports clustering, that in conjunction with Queue Mirroring can be used for an Active-Active configuration as explained in the Clustering Guide and in the Highly Available (Mirrored) Queues article, which includes steps on how to set up the clusters and monitor queues. The clustered instances should be fronted by a TCP Load Balancer such as HAProxy, and clients should connect to the cluster using the address of the proxy. For more information, see the Multi-tenancy support in FortiSOAR guide.

Note

For the purpose of the following procedure, we are considering a two-node MQ mirrored queue clusters that are both added to the Reverse Proxy.

  1. Configure the Reverse Proxy to pass requests only to Node1, which is the primary node of the MQ cluster.
    Therefore, now all requests will be handled by Node1 and Node2 will be available for maintenance.
  2. Log on to the Node2 terminal session as a root user, and upgrade Node2 by following the steps mentioned in the Upgrading a FortiSOAR Secure Message Exchange section.
  3. Configure the Reverse Proxy to route requests through Node2. Therefore, now all requests will be handled by Node2 and Node1 will be available for maintenance.
  4. Login to Node1, and upgrade Node1 as per the procedure mentioned in step 2.
  5. Reconfigure the Reverse Proxy to load balance both Node1 and Node2.

Upgrading a FortiSOAR Distributed Multi-Tenancy Configuration

This section describes the procedure to upgrade a FortiSOAR distributed multi-tenant configuration for managed security services providers (MSSPs) or Distributed SOC configuration.

You must first upgrade the master node of your FortiSOAR distributed multi-tenant configuration and only then upgrade the tenant nodes of your FortiSOAR multi-tenancy setup.

Caution

In case of a distributed deployment, both the master and the tenant nodes must be upgraded. A version mismatch will not work if either of them upgrades to 7.2.2.

Upgrading a FortiSOAR master node

Before you upgrade your FortiSOAR master node, ensure the following:

  • All playbooks have completed their execution on the master.
  • The tenant node(s) are deactivated from the master node before upgrading the master node.

If the master node of your multi-tenant configuration is part of an HA setup, i.e., MSSP +HA, then follow the steps mentioned in the Upgrading a FortiSOAR High Availability Cluster section.

If the master node of your multi-tenant configuration is not part of an HA setup, then follow the steps mentioned in the Upgrading a FortiSOAR Enterprise Instance section.

Upgrading a FortiSOAR Tenant node

Before you upgrade your FortiSOAR tenant node, ensure the following:

  • Data replication from the tenant node to the master node is stopped. You can stop data replication by logging on to the tenant node and clicking Settings to open the System page, then in the Multi Tenancy section, click the Master Configuration menu item and then in the Communication With Master Node section, toggle the Enabled button to NO.
  • All playbooks have completed their execution on the tenant.
  • All schedule playbooks that fetch data from data sources to the tenant are stopped.
  • Any application that pushes data from data sources to the tenant is stopped.

If the tenant node of your multi-tenant configuration is part of an HA setup, i.e., MSSP +HA, then follow the steps mentioned in the Upgrading a FortiSOAR High Availability Cluster section.

If the tenant node of your multi-tenant configuration is not part of an HA setup, then follow the steps mentioned in the Upgrading a FortiSOAR Enterprise Instance section.

Note After the tenant node has been successfully upgraded, you must toggle the Allow Module Management setting to NO and then back to YES. This is needed only if you were already using the 'Allow Module Management' feature and is required to synchronize the tenant module metadata with the master instance. You can ignore this step, if your 'Allow Module Management' setting was already disabled before the upgrade.

Upgrading a FortiSOAR Secure Message Exchange

A secure message exchange establishes a secure channel that is used to relay information to the agents or tenant nodes. To create a dedicated secure channel, you are required to add the reference of the installed and configured secure message exchange, when you add agent or tenant nodes to your environment. For information on agents see the Segmented Network support in FortiSOAR chapter in the "Administration Guide," and for more information on secure message exchange and tenants, see the "Multi-Tenancy support in FortiSOAR Guide".

  1. Ensure that you stop data replication between the master and the tenant nodes. You can stop data replication by logging on to the tenant node and clicking Settings to open the System page, then in the Multi Tenancy section, click the Master Configuration menu item and then in the Communication With Master Node section, toggle the Enabled button to NO.
  2. SSH to the secure message exchange VM that you want to upgrade.
  3. Check that you are connected to a screen session. A screen session is needed for situations where network connectivity is less than favorable. You can check your screen session using the following command:
    # screen -ls
    This command returns an output such as the following example:
    There is a screen on:
    12081.upgrade(Detached)

    Log back into the SSH console and run the following command to reattach the screen session:
    screen -r 12081.upgrade
    OR
    screen -r upgrade
  4. Run the following command to download the upgrade installer:
    # wget https://repo.fortisoar.fortinet.com/7.2.2/upgrade-fortisoar-7.2.2.bin
  5. Run the upgrade installer using the following command:
    # sh upgrade-fortisoar-7.2.2.bin
    OR
    # chmod +x upgrade-fortisoar-7.2.2.bin
    # ./upgrade-fortisoar-7.2.2.bin
    Notes: The FortiSOAR upgrade installer checks for the following:
    • The space available in /tmp or /var/temp (if exist in /etc/fstab), which must be at least 2GB. If the space available in in /tmp or /var/temp is less than 2GB, then the upgrade installer exits after displaying an appropriate error message. If you want to skip this space validation check, you can use the --skip-tmp-validation option while running the upgrade script:
      # ./upgrade-fortisoar-7.2.2.bin --skip-tmp-validation
    • The disk space available in /boot, and if the /boot has insufficient space, then the upgrade installer exits after displaying an appropriate error message. Steps for cleaning up /boot are present in the Clean up /boot article present in the Fortinet Knowledge Base.
    • The disk space available in /var/lib/pgsql to ensure that you have sufficient disk space for pgsql. If you do not have sufficient disk space for pgsql, in this case also the upgrade installer exits. In these cases, you must increase the partition size for /var/lib/pgsql. For the procedure to increase the partition size, see the 'Issues occurring in FortiSOAR due to insufficient space' section in the Deployment Troubleshooting chapter in the "Deployment Guide" for more information.
    Once you complete cleaning up /boot and/or increasing disk space and space in /tmp (as per the messages provided by the upgrade installer) and you have sufficient space for upgrading FortiSOAR, you must re-run the upgrade installer to continue the process of upgrading FortiSOAR.
  6. Once you have successfully upgraded the secure message exchange, start the data replication between the master and the tenant nodes again by toggling the Data Replication button to ON, and then verify the replication.

Upgrading a FortiSOAR Secure Message Exchange Cluster

RabbitMQ supports clustering, that in conjunction with Queue Mirroring can be used for an Active-Active configuration as explained in the Clustering Guide and in the Highly Available (Mirrored) Queues article, which includes steps on how to set up the clusters and monitor queues. The clustered instances should be fronted by a TCP Load Balancer such as HAProxy, and clients should connect to the cluster using the address of the proxy. For more information, see the Multi-tenancy support in FortiSOAR guide.

Note

For the purpose of the following procedure, we are considering a two-node MQ mirrored queue clusters that are both added to the Reverse Proxy.

  1. Configure the Reverse Proxy to pass requests only to Node1, which is the primary node of the MQ cluster.
    Therefore, now all requests will be handled by Node1 and Node2 will be available for maintenance.
  2. Log on to the Node2 terminal session as a root user, and upgrade Node2 by following the steps mentioned in the Upgrading a FortiSOAR Secure Message Exchange section.
  3. Configure the Reverse Proxy to route requests through Node2. Therefore, now all requests will be handled by Node2 and Node1 will be available for maintenance.
  4. Login to Node1, and upgrade Node1 as per the procedure mentioned in step 2.
  5. Reconfigure the Reverse Proxy to load balance both Node1 and Node2.