HA heartbeat & active node election

HA heartbeat

You can group multiple FortiWeb appliances together as a high availability (HA) group (see FortiWeb high availability (HA) ). The heartbeat traffic indicates to other appliances in the HA group that the appliance is up and “alive.”

Heartbeat traffic between HA members occurs over the physical network ports selected in Heartbeat Interface. Heartbeat traffic uses multicast on port number 6065 and the IP address 239.0.0.1. The HA IP addresses are hard-coded and cannot be modified.

Ensure that switches and routers that connect to heartbeat interfaces are configured to allow level2 frames. See Heartbeat packet Ethertypes.

Failover is triggered by any interruption to either the heartbeat or a port monitored network interface whose length of time exceeds your configured limits (Detection Interval and Heartbeat Lost Threshold). When the active (or primary) appliance becomes unresponsive, the standby (or secondary) appliance:

1. Assumes the virtual MAC address of the failed primary unit and broadcasts ARP/NS packets so that other equipment in the network will refresh their MAC forwarding tables and detect the new primary unit
2. Assumes the role of the active appliance and scans network traffic

The heartbeat timeout is calculated by:

Heartbeat timeout = Detection Interval x Heartbeat Lost Threshold

Time required for traffic to be redirected to the new active appliance varies by your network’s responsiveness to changeover notification and by your configuration:

Total failover time = ARP/NS Packet Numbers x ARP/NS Packet Interval(sec) + Network responsiveness + Heartbeat timeout

For example, if:

• Detection Interval is 3 (i.e. 0.3 seconds)
• Heartbeat Lost Threshold is 2
• ARP/NS Packet Numbers is 3
• ARP/NS Packet Interval (sec) is 1
• Network switches etc. take 2 seconds to acknowledge and redirect traffic flow

then the total time between the first unacknowledged heartbeat and traffic redirection could be up to 5.6 seconds.

The above settings can be configured in the CLI using the system ha command. For details, see FortiWeb CLI Reference.

Heartbeat packet Ethertypes

Normal IP packets are 802.3 packets that have an Ethernet type (Ethertype) field value of 0x0800. Ethertype values other than 0x0800 are understood as level2 frames rather than IP packets.

By default, HA uses the following Ethertypes:

• Ethertype 0x8890—For HA heartbeat packets that HA members use to find other member and to verify the status of other members while the HA group is operating.
• Ethertype 0x8893—For HA sessions that synchronize the HA configurations.

Because heartbeat packets are recognized as level2 frames, the switches and routers that connect to heartbeat interfaces require a configuration that allows them. If these network devices drop level2 frames, they prevent heartbeat traffic between the members of the HA group.

In some cases, if you connect and configure the heartbeat interfaces so that regular traffic flows but heartbeat traffic is not forwarded, you can change the configuration of the switch that connects the HA heartbeat interfaces to allow level2 frames with Ethertypes 0x8890 and 0x8893 to pass.

For HA Ethertype, only numbers between 0x8890–0x889f can be used; also, different HA Ethertype shall use different numbers.

How HA chooses the active appliance

Members in an HA group may or may not resume their active and standby roles when the failed appliance resumes responsiveness to the heartbeat.

Since the current active appliance will by definition have a greater uptime than a failed previous active appliance that has just returned online, assuming each has the same number of available ports, the current active appliance usually retains its status as the active appliance, unless Override is enabled. If Override is enabled, and if Device Priority of the returning appliance is higher, it will be elected as the active appliance in the HA group.

If Override is disabled, HA considers (in order):

1. The most available ports

For example, if two FortiWeb appliances, FortiWeb1 and FortiWeb2, are configured to monitor two ports each, and FortiWeb2 has only one port currently available according to Port Monitor, FortiWeb1 would become the active appliance, regardless of uptime or priority. But if both have 2 available ports, this factor alone would not be able to determine which appliance should be active, and the HA group would proceed to the next consideration.

2. The highest uptime value
   Uptime is reset to zero if an appliance fails. Sometimes the status change of the monitored ports may also lead to uptime being reset to 0.
3. The smallest Device Priority number (that is, 0 has the highest priority)
4. The highest-sorting serial number

Serial numbers are sorted by comparing each character from left to right, where 9 and z are the greatest values, and result in highest placement in the sorted list.

If Override is enabled, HA considers (in order):

1. The most available ports
2. The smallest Device Priority number (that is, 0 has the highest priority)
3. The highest uptime value
   Uptime is reset to zero if an appliance fails. Sometimes the status change of the monitored ports may also lead to uptime being reset to 0.
4. The highest-sorting serial number

If the heartbeat link occurs through switches or routers, and the active appliance is very busy, it might require more time to establish a heartbeat link through which it can negotiate to elect the active appliance. You can configure the amount of time that a FortiWeb appliance will wait after it boots to establish this connection before assuming that the other appliance is unresponsive, and that it should become the active appliance. For details, see the boot-time <seconds_int> setting in FortiWeb CLI Reference.

See also

• FortiWeb high availability (HA)
• Replicating the configuration without FortiWeb HA (external HA)