Azure SDN connector moves private IP address on trusted NIC during A-P HA failover 7.4.5

This feature introduces a floating private IP address on the trusted NIC (port2).

In earlier FortiOS versions, the SDN connector is leveraged to reassociate the public IP address from the old primary instance untrust interface to the new primary instance untrust interface during active-passive high availability (A-P HA) failover and change the next hop to redirect traffic to the new primary instance. In this scenario, you needed to manually update user-defined routes (UDR) to redirect traffic to the trusted NIC on the new primary instance, which became laborious and difficult to manage.

This feature allows you to avoid manually updating the UDRs after failover. Instead, you can configure all UDRs to use the secondary floating IP address as the next hop. When failover occurs, the SDN connector switches the secondary floating private IP address from the old primary instance to the new primary instance. You can achieve this by deleting the secondary private IP interface and recreating it with the same IP address on the new primary instance. UDRs can remain unchanged. Failover duration depends on the time taken to reassign the private IP address.

The following shows the topology for an example deployment:

The following instructions assume that you have already deployed FortiGate-VMs on Azure as an A-P HA cluster, with the following ports configuration:

Port	Description
port1	untrusted, to_Internet
port2	trusted
port3	HA-sync
port4	HA-mgmt

The SDN connector cannot update the elastic IP address (EIP) without valid authentication. For the SDN connector authentication, you can configure one of the following:

• Azure SDN connector service principal configuration requirements
• Configuring an SDN connector using a managed identity

The following example uses managed identities.

To configure Azure SDN connector to move private IP address on trusted NIC during A-P HA failover:

1. Assign the contributor role to the HA cluster nodes with a scope. See Configure managed identities on Azure virtual machines (VMs):

   SCOPE="$(az group show -g $RG --query "id" -otsv)"
   az vm identity assign --resource-group $RG --role Contributor --scope $SCOPE --name VNET0-FGT-A
   az vm identity assign --resource-group $RG --role Contributor --scope $SCOPE --name VNET0-FGT-B
   
   {
     "role": "Contributor",
     "scope": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A",
     "systemAssignedIdentity": "4ae41c9a-146a-415b-b8f8-0a8fdffa6ad8",
     "userAssignedIdentities": {}
   }
   {
     "role": "Contributor",
     "scope": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A",
     "systemAssignedIdentity": "9bb34ee6-ae9b-42a1-8d9a-084133ba3b0f",
     "userAssignedIdentities": {}
   }

2. Verify that the SDN connector is configured and can update the EIP:

   VNET0-FGT-B (Interim)# get system status
   Version: FortiGate-VM64-AZURE v7.4.5,build2686,240806 (interim)
   First GA patch build date: 230509
   Security Level: 0
   Firmware Signature: not-certified
   Virus-DB: 1.00000(2018-04-09 18:07)
   Extended DB: 1.00000(2018-04-09 18:07)
   Extreme DB: 1.00000(2018-04-09 18:07)
   AV AI/ML Model: 0.00000(2001-01-01 00:00)
   IPS-DB: 6.00741(2015-12-01 02:30)
   IPS-ETDB: 6.00741(2015-12-01 02:30)
   APP-DB: 6.00741(2015-12-01 02:30)
   Proxy-IPS-DB: 6.00741(2015-12-01 02:30)
   Proxy-IPS-ETDB: 6.00741(2015-12-01 02:30)
   Proxy-APP-DB: 6.00741(2015-12-01 02:30)
   FMWP-DB: 24.00071(2024-07-31 17:46)
   IPS Malicious URL Database: 1.00001(2015-01-01 01:01)
   IoT-Detect: 0.00000(2022-08-17 17:31)
   OT-Detect-DB: 0.00000(2001-01-01 00:00)
   OT-Patch-DB: 0.00000(2001-01-01 00:00)
   OT-Threat-DB: 6.00741(2015-12-01 02:30)
   IPS-Engine: 7.00539(2024-05-09 00:34)
   Serial-Number: FGTAZRLF5HB4I_03
   License Status: Valid
   VM Resources: 4 CPU, 7978 MB RAM
   Log hard disk: Available
   Hostname: VNET0-FGT-B
   Private Encryption: Disable
   Operation Mode: NAT
   Current virtual domain: root
   Max number of virtual domains: 2
   Virtual domains status: 1 in NAT mode, 0 in TP mode
   Virtual domain configuration: disable
   FIPS-CC mode: disable
   Current HA mode: a-p, primary
   Cluster uptime: 47 minutes, 22 seconds
   Cluster state change time: 2024-08-06 12:12:00
   Branch point: 2686
   Release Version Information: interim
   FortiOS x86-64: Yes
   System time: Tue Aug  6 12:39:13 2024
   Last reboot reason: warm reboot
   
   VNET0-FGT-B (Interim)# show system sdn-connector
   config system sdn-connector
       edit "AzureSDN"
           set type azure
           set ha-status enable
           set subscription-id "4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a"
           set resource-group "6899_HA-A"
           config nic
               edit "VNET0-FGT-B-Nic1"
                   config ip
                       edit "ipconfig1"
                           set public-ip "HA-A-PIP"
                       next
                   end
               next
           end
           config route-table
               edit "VNET0-RouteTable-ProtectedSubnet"
                   config route
                       edit "toDefault"
                           set next-hop "172.16.32.21"
                       next
                   end
               next
           end
       next
   end
   
   VNET0-FGT-B (Interim)# diagnose sys sdn status
   SDN Connector                       Type        Status
   -------------------------------------------------------------
   AzureSDN                            azure       Up
   
   VNET0-FGT-B (Interim)# execute update-eip
   NIC: 172.16.32.5, public IP: 20.191.71.72
   NIC: 172.16.32.21
   NIC: 172.16.32.37
   NIC: 172.16.32.53, public IP: 13.66.252.150
   port1: 172.16.32.5, eip: 20.191.71.72
   port2: 172.16.32.20
   EIP is updated successfully

3. On both HA nodes, configure the SDN connector as follows. The commands configure new options, peer-nic and private-ip:
   

   # config for VNET0-FGT-A
   #
   config system sdn-connector
       edit "AzureSDN"
           config nic
               edit "VNET0-FGT-A-Nic2"
                   set peer-nic "VNET0-FGT-B-Nic2"
                   config ip
                       edit "ipconfig2"
                           set private-ip "172.16.32.22"
                       next
                   end
               next
           end
           config route-table
               edit "VNET0-RouteTable-ProtectedSubnet"
                   config route
                       edit "toDefault"
                           set next-hop "172.16.32.22"
                       next
                   end
               next
           end
       next
   end
   
   # config for VNET0-FGT-B
   #
   config system sdn-connector
       edit "AzureSDN"
           config nic
               edit "VNET0-FGT-B-Nic2"
                   set peer-nic "VNET0-FGT-A-Nic2"
                   config ip
                       edit "ipconfig2"
                           set private-ip "172.16.32.22"
                       next
                   end
               next
           end
           config route-table
               edit "VNET0-RouteTable-ProtectedSubnet"
                   config route
                       edit "toDefault"
                           set next-hop "172.16.32.22"
                       next
                   end
               next
           end
       next
   end

4. Add a secondary IP address, 172.16.32.22 in this example, to the HA nodes' port2:

   config system interface
       edit "port2"
           set secondary-IP enable
           config secondaryip
               edit 1
                   set ip 172.16.32.22/28
                   set allowaccess ping
               next
           end
       next
   end

5. Run the following in the Azure CLI to associate a secondary IP address to port2:

   az network nic ip-config create --resource-group $RG \
       --nic-name VNET0-FGT-B-NIC2 --name ipconfig2 \
       --private-ip-address 172.16.32.22
   {
     "etag": "W/\"c0249b51-62fe-4524-9676-4b58a167f244\"",
     "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/networkInterfaces/VNET0-FGT-B-Nic2/ipConfigurations/ipconfig2",
     "name": "ipconfig2",
     "primary": false,
     "privateIPAddress": "172.16.32.22",
     "privateIPAddressVersion": "IPv4",
     "privateIPAllocationMethod": "Static",
     "provisioningState": "Succeeded",
     "resourceGroup": "6899_HA-A",
     "subnet": {
       "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/virtualNetworks/VNET0/subnets/InternalSubnet",
       "resourceGroup": "6899_HA-A"
     },
     "type": "Microsoft.Network/networkInterfaces/ipConfigurations"
   }

6. Verify the IP configurations on port2 of both nodes:

   az network nic ip-config list -g $RG  --nic-name VNET0-FGT-B-Nic2
   [
     {
       "etag": "W/\"c0249b51-62fe-4524-9676-4b58a167f244\"",
       "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/networkInterfaces/VNET0-FGT-B-Nic2/ipConfigurations/ipconfig1",
       "name": "ipconfig1",
       "primary": true,
       "privateIPAddress": "172.16.32.21",
       "privateIPAddressVersion": "IPv4",
       "privateIPAllocationMethod": "Static",
       "provisioningState": "Succeeded",
       "resourceGroup": "6899_HA-A",
       "subnet": {
         "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/virtualNetworks/VNET0/subnets/InternalSubnet",
         "resourceGroup": "6899_HA-A"
       },
       "type": "Microsoft.Network/networkInterfaces/ipConfigurations"
     },
     {
       "etag": "W/\"c0249b51-62fe-4524-9676-4b58a167f244\"",
       "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/networkInterfaces/VNET0-FGT-B-Nic2/ipConfigurations/ipconfig2",
       "name": "ipconfig2",
       "primary": false,
       "privateIPAddress": "172.16.32.22",
       "privateIPAddressVersion": "IPv4",
       "privateIPAllocationMethod": "Static",
       "provisioningState": "Succeeded",
       "resourceGroup": "6899_HA-A",
       "subnet": {
         "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/virtualNetworks/VNET0/subnets/InternalSubnet",
         "resourceGroup": "6899_HA-A"
       },
       "type": "Microsoft.Network/networkInterfaces/ipConfigurations"
     }
   ]

7. In the Azure CLI, verify that the route toDefault next hop IP address is 172.16.32.22:

   az network route-table route show -g $RG --route-table-name VNET0-RouteTable-ProtectedSubnet
    --name toDefault
   {
     "addressPrefix": "0.0.0.0/0",
     "etag": "W/\"879c8971-1b10-4905-83fd-b63e1c8f76c7\"",
     "hasBgpOverride": false,
     "id": "/subscriptions/4f27b38c-ad3f-43d8-a9a3-01182e5e2f9a/resourceGroups/6899_HA-A/providers/Microsoft.Network/routeTables/VNET0-RouteTable-ProtectedSubnet/routes/toDefault",
     "name": "toDefault",
     "nextHopIpAddress": "172.16.32.22",
     "nextHopType": "VirtualAppliance",
     "provisioningState": "Succeeded",
     "resourceGroup": "6899_HA-A",
     "type": "Microsoft.Network/routeTables/routes"
   }

8. Configure the following so that the endpoint can reach the internet:

   config firewall policy
       edit 100
           set name "to_Internet"
           set srcintf "port2"
           set dstintf "port1"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
           set nat enable
       next
   end
   
   config firewall vip
       edit "172.16.33.4:80"
           set mappedip "172.16.33.4"
           set extintf "port1"
           set portforward enable
           set extport 80
           set mappedport 80
       next
       edit "172.16.33.4:443"
           set mappedip "172.16.33.4"
           set extintf "port1"
           set portforward enable
           set extport 443
           set mappedport 443
       next
       edit "172.16.33.4:65122"
           set mappedip "172.16.33.4"
           set extintf "port1"
           set portforward enable
           set extport 65122
           set mappedport 22
       next
       edit "172.16.33.4:69"
           set mappedip "172.16.33.4"
           set extintf "port1"
           set portforward enable
           set protocol udp
           set extport 69
           set mappedport 69
       next
   end
   config firewall vipgrp
       edit "VIPs_on_Internal"
           set interface "port1"
           set member "172.16.33.4:443" "172.16.33.4:80" "172.16.33.4:65122" "172.16.33.4:69"
       next
   end
   config firewall policy
       edit 200
           set name "VIP"
           set srcintf "port1"
           set dstintf "port2"
           set action accept
           set srcaddr "all"
           set dstaddr "VIPs_on_Internal"
           set schedule "always"
           set service "ALL"
       next
   end
   
   config system interface
   edit root_lo0
   set vdom root
   set type loopback
   end
   config firewall policy
       edit 1000
           set srcintf "root_lo0"
           set dstintf "root_lo0"
           set srcaddr "all"
           set dstaddr "all"
           set srcaddr6 "all"
           set dstaddr6 "all"
           set action accept
           set schedule "always"
           set service "ALL"
           set utm-status enable
           set ssl-ssh-profile "deep-inspection"
           set av-profile "default"
           set webfilter-profile "default"
           set dnsfilter-profile "default"
           set file-filter-profile "default"
           set ips-sensor "default"
           set application-list "default"
       next
   end

9. Verify that the endpoint can reach the internet.
10. Trigger HA failover.
11. Verify that the endpoint can reach the internet after failover.