HA cluster out-of-band management

To provide direct management access to all cluster units, a management interface can be reserved as port of the HA configuration. After the management interface is reserved, the various interface settings, such as the IP address and administrative access, can be configured for each individual cluster unit. These settings are not synchronized to other cluster units. Connect the management interface of each cluster unit to your network to individually manage each unit with its unique IP address.

• Reserved management interfaces provide direct management access to each cluster unit, and give each cluster unit a different identity on your network. This simplifies using external services, such as SNMP, to monitor and manage each cluster units.

• Reserved management interfaces are not assigned HA virtual MAC addresses. They retain the permanent hardware address of the physical interface, unless you manually change it using the config system interface command.

• Reserved management interfaces and their IP addresses should not be used for managing a cluster using FortiManager. To manage a FortiProxy HA cluster with FortiManager, use the IP address of one of the cluster unit interfaces.

• Configuration changes to a reserved management interface are not synchronized to other cluster units. Other configuration changes are automatically synchronized to all cluster units.

For outgoing traffic, the management interface supports only the following protocols: SNMP, NTP, LOG, Radius, FTP, TFTP, telnet. For the management interface to handle outgoing traffic with an unsupported protocol, you must configure multiple VDOMs and dedicate the root VDOM to management traffic, which means assigning the management interface and the ha-sync interface to the root VDOM while keeping all other interfaces for in-band traffic to user VDOMs.

Management interface

Enable HTTPS or HTTP administrative access on the reserved management interfaces to connect to the GUI of each cluster unit. On secondary units, the GUI has the same features as the primary unit, except for unit specific information, for example:

• The System Information widget on the Dashboard shows the secondary unit's serial number, and also the same cluster information as on the primary unit.

• In the cluster list at System > HA, you can change the HA configuration for the unit that you are logged into, but you can only change the host name and device priority for the primary unit and other secondary units.

• The system events logs show logs for the device that you are logged into. To view logs for other cluster units, including the primary unit, use the cluster member list at System > HA.

Enable SNMP administrative access on a reserved management interface to use SNMP to monitor each cluster unit using the interface's IP address. Direct management of cluster members must also be enabled, see Configuration example.

Enable SSH or TELNET administrative access on the reserved management interfaces to connect to the CLI of each cluster unit. The CLI prompt includes the host of the cluster unit that you are connected to. Use the execute ha manage command to connect to other cluster unit CLIs.

SNMP, remote authentication server and other management services

By default, management services such as SNMP, remote authentication (LDAP, RADIUS, TACACS+, and others), remote logging, and others use a cluster interface. This means that communication from each cluster unit comes from a cluster interface, and not from an individual cluster unit's interface or the HA reserved management interface.

You can configure HA reserved management interfaces to be used for communication with management services by enabling the ha-direct option. This separates management traffic for each cluster unit, and allows each unit to be individually managed. This is especially useful when cluster units are in different physical locations.

The following management features will then use the HA reserved management interface:

• SNMP queries and traps

• Remote authentication and certificate verification

• Communication with FortiSandbox

• Remote logging

To use the HA reserved management interface for management:

config system ha
    set ha-direct enable
end

SNMP requires ha-direct to be configured under SNMP settings only. See below for more configuration options.

Configuration example

This example describes how to configure SNMP remote management of individual cluster units using an HA reserved management interface. The configuration consists of two FortiProxy units already operating as a cluster.

Two FortiProxy units are already operating in a cluster. On each unit, port8 is connected to the internal network through a switch and configured as an out-of-band reserved management interface.

Administrative access and default route for HA management interface

To configure the primary unit's reserved management interface, configure an IP address and management access on port8. Then, configure the necessary HA settings to enable the HA reserved management interface and its route.

To configure the secondary unit's reserved management interface, access the unit's CLI through the primary unit, and configure an IP address, management access on port8, and the necessary HA settings. Configuration changes to the reserved management interface are not synchronized to other cluster units.

To configure the primary unit reserved management interface in CLI:

1. From a computer on the internal network, connect to the CLI at 10.11.101.100 on port2.

2. Change the port8 IP address and management access:

   config system interface
       edit port8
           set ip 10.11.101.101/24
           set allowaccess https ping ssh snmp
       next
   end

3. Configure the HA settings for the HA reserved management interface by defining a default route to route to the gateway 10.11.101.2:

   config system ha
       set ha-mgmt-status enable
       config ha-mgmt-interfaces
           edit 1
               set interface port8
               set gateway 10.11.101.2
               set gateway6 2001:db8:0:2::20
           next
       end
   end

You can now log into the primary unit's GUI by browsing to https://10.11.101.101. You can also log into the primary unit's CLI by using an SSH client to connect to 10.11.101.101.

To configure secondary unit reserved management interfaces to allow HTTPS, SSH, and ICMP access:

1. From a computer on the internal network, connect to the primary unit's CLI.

2. Connect to the secondary unit with the following command:

   execute ha manage <unit id> <username> <password>
   

3. Change the port8 IP address and management access:

   config system interface
       edit port8
           set ip 10.11.101.102/24
           set allowaccess https ping ssh snmp
       next
   end

4. Configure the HA settings for the HA reserved management interface by defining a default route to route to the gateway 10.11.101.2:

   config system ha
       set ha-mgmt-status enable
       config ha-mgmt-interfaces
           edit 1
               set interface port8
               set gateway 10.11.101.2
               set gateway6 2001:db8:0:2::20
           next
       end
   end

You can now log into the secondary unit's GUI by browsing to https://10.11.101.102. You can also log into the secondary unit's CLI by using an SSH client to connect to 10.11.101.102.

SNMP monitoring

The SNMP server can get status information from the cluster members. To use the reserved management interfaces, you must add at least one HA direct management host to an SNMP community. If the SNMP configuration includes SNMP users with user names and passwords, HA direct management must be enabled for the users. The SNMP configuration is synchronized to all cluster units.

To configure the cluster for SNMP management using the reserved management interfaces:

1. Add an SNMP community with a host for the reserved management interface of each cluster member. The host includes the IP address of the SNMP server.

   config system snmp community
       edit 1
           set name "Community"
           config hosts
               edit 1
                   set ip 10.11.101.20 255.255.255.255
                   set ha-direct enable
               next
           end
       next
   end

   Enabling ha-direct in a non-HA environment will make SNMP unusable.

2. Add an SNMP user for the reserved management interface:

   config system snmp user
       edit "1"
           set notify-hosts 10.11.101.20
           set ha-direct enable
       next
   end

3. Configure remaining settings as required.

To get CPU, memory, and network usage information from the SNMP manager for each cluster unit using the reserved management IP addresses:

1. Connect to the SNMP manager CLI.

2. Get resource usage information for the primary unit using the MIB fields:

   snmpget -v2c -c Community 10.11.101.101 fgHaStatsCpuUsage
   snmpget -v2c -c Community 10.11.101.101 fgHaStatsMemUsage
   snmpget -v2c -c Community 10.11.101.101 fgHaStatsNetUsage
   

3. Get resource usage information for the primary unit using the OIDs:

   snmpget -v2c -c Community 10.11.101.101 1.3.6.1.4.1.12356.101.13.2.1.1.3.1
   snmpget -v2c -c Community 10.11.101.101 1.3.6.1.4.1.12356.101.13.2.1.1.4.1
   snmpget -v2c -c Community 10.11.101.101 1.3.6.1.4.1.12356.101.13.2.1.1.5.1
   

4. Get resource usage information for the secondary unit using the MIB fields:

   snmpget -v2c -c Community 10.11.101.102 fgHaStatsCpuUsage
   snmpget -v2c -c Community 10.11.101.102 fgHaStatsMemUsage
   snmpget -v2c -c Community 10.11.101.102 fgHaStatsNetUsage
   

5. Get resource usage information for the primary unit using the OIDs:

   snmpget -v2c -c Community 10.11.101.102 1.3.6.1.4.1.12356.101.13.2.1.1.3.1
   snmpget -v2c -c Community 10.11.101.102 1.3.6.1.4.1.12356.101.13.2.1.1.4.1
   snmpget -v2c -c Community 10.11.101.102 1.3.6.1.4.1.12356.101.13.2.1.1.5.1
   

Remote authentication over dedicated HA management interface

If a dedicated management interface is configured on each cluster unit, and authentication on a remote server is used (administrator log in, firewall authentication, SSL VPN access, and so on), then the FortiProxy units can contact the remote authentication server using the HA management interface if ha-direct is enabled.

This shows an example configuration for a remote system administrator:

config system ha
    set group-id 110
    set group-name "leo_fpx_ha_11"
    set mode active-passive
    set hbdev "port5" 50
    set ha-mgmt-status enable
    config ha-mgmt-interfaces
        edit 1
            set interface "port4"
            set gateway 10.150.0.3
        next
    end
    set override enable
    set priority 111
    set unicast-hb enable
    set unicast-hb-peerip 10.150.51.12
end

config system admin
    edit "leo_rad"
        set remote-auth enable
        set accprofile "prof_admin"
        set remote-group "radius_grp_mgmt"
    next
end

config user radius
    edit "radius_mgmt"
        set server "10.150.0.111"
        set secret ************
    next
end

config user group
    edit "radius_grp_mgmt"
        set member "radius_mgmt"
    next
end