Deploying FortiGate-VM HA on GCP between multiple zones

This guide provides a sample deployment of active-passive FortiGate-VM high availability (HA) on GCP between multiple zones:

1. Check the prerequisites before deployment.
2. Create FortiGate A in one zone as the primary FortiGate, using metadata that has the ha-master configuration.
3. Create FortiGate B in another zone as the secondary FortiGate, using metadata that has the ha-slave configuration.
4. Create an Ubuntu PC which can access the Internet via FortiGate HA.
5. Shut down FortiGate A. FortiGate B becomes the primary FortiGate and handles the traffic, and the public external IP address attaches to FortiGate B.
6. Configure a VDOM exception.
7. Run a diagnose command to see what happened to the route and public external IP address during the failover procedure.

The following depicts the network topology for this sample deployment:

IPsec VPN phase 1 configuration does not synchronize between primary and secondary FortiGates across zones. Phase 2 configuration does synchronize.

To check the prerequisites:

• Ensure that you have created four VPC networks.
• Ensure that you have created routes for each network.
• Create firewall rules for each network.
• Reserving three external IP addresses is suggested for convenience.

To create FortiGate A in one zone as the primary FortiGate using metadata that has the ha-master configuration:

This example creates FortiGate A in zone c.

1. Run the following commands in GCP:

   gcloud beta compute --project=dev-project-001-166400 instances create fgt-a --zone=us-central1-c --machine-type=n1-standard-4 --network-tier=PREMIUM --can-ip-forward --maintenance-policy=MIGRATE --service-account=966517025500-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --image=ond-0804 --image-project=dev-project-001-166400 --boot-disk-type=pd-standard --boot-disk-device-name=fgt-0804 --network-interface subnet=hapvc-port1external,private-network-ip=10.0.0.15,address=104.154.241.0 --network-interface subnet=hapvc-port2internal,private-network-ip=10.0.1.15,no-address --network-interface subnet=hapvc-port3heartbeat,private-network-ip=10.0.2.15,no-address --network-interface subnet=hapvc-port4mgmt,private-network-ip=10.0.3.15,address=104.154.25.116 --metadata-from-file user-data=/home/gcloud/config/master.conf

2. Run the following commands in FortiOS:

   config system ha

   set group-id 21

   set group-name "cluster1"

   set mode a-p

   set hbdev "port3" 50

   set session-pickup enable

   set session-pickup-connectionless enable

   set ha-mgmt-status enable

   config ha-mgmt-interfaces

   edit 1

   set interface "port4"

   set gateway 10.0.3.1

   next

   end

   set override enable

   set priority 200

   set unicast-hb enable

   set unicast-hb-peerip 10.0.2.16

   set unicast-hb-netmask 255.255.255.0

   end

   config system sdn-connector

   edit "gcp_conn"

   set type gcp

   set ha-status enable

   config external-ip

   edit "reserve-fgthapublic"

   next

   end

   config route

   edit "route-internal"

   next

   end

   set use-metadata-iam disable

   set gcp-project "..."

   set service-account "..."

   set private-key "..."

   next

   end

To create FortiGate B in another zone as the secondary FortiGate using metadata that has the ha-slave configuration:

This example creates FortiGate B in zone a.

1. Run the following commands in GCP:

   gcloud beta compute --project=dev-project-001-166400 instances create fgt-b --zone=us-central1-a --machine-type=n1-standard-4 --network-tier=PREMIUM --can-ip-forward --maintenance-policy=MIGRATE --service-account=966517025500-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --image=ond-0804 --image-project=dev-project-001-166400 --boot-disk-type=pd-standard --boot-disk-device-name=fgt-0804 --network-interface subnet=hapvc-port1external,private-network-ip=10.0.0.16,no-address --network-interface subnet=hapvc-port2internal,private-network-ip=10.0.1.16,no-address --network-interface subnet=hapvc-port3heartbeat,private-network-ip=10.0.2.16,no-address --network-interface subnet=hapvc-port4mgmt,private-network-ip=10.0.3.16,address=35.226.235.236 --metadata-from-file user-data=/home/gcloud/config/slave.conf

2. Run the following commands in FortiOS:

   config system ha

   set group-id 21

   set group-name "cluster1"

   set mode a-p

   set hbdev "port3" 50

   set session-pickup enable

   set session-pickup-connectionless enable

   set ha-mgmt-status enable

   config ha-mgmt-interfaces

   edit 1

   set interface "port4"

   set gateway 10.0.3.1

   next

   end

   set override enable

   set priority 200

   set unicast-hb enable

   set unicast-hb-peerip 10.0.2.15

   set unicast-hb-netmask 255.255.255.0

   end

To create an Ubuntu PC that can access the Internet via FortiGate HA:

Run the following commands in GCP:

gcloud beta compute --project=dev-project-001-166400 instances create fgt-b --zone=us-central1-a --machine-type=n1-standard-4 --network-tier=PREMIUM --can-ip-forward --maintenance-policy=MIGRATE --service-account=966517025500-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/cloud-platform --image=ond-0804 --image-project=dev-project-001-166400 --boot-disk-type=pd-standard --boot-disk-device-name=fgt-0804 --network-interface subnet=hapvc-port1external,private-network-ip=10.0.0.16,no-address --network-interface subnet=hapvc-port2internal,private-network-ip=10.0.1.16,no-address --network-interface subnet=hapvc-port3heartbeat,private-network-ip=10.0.2.16,no-address --network-interface subnet=hapvc-port4mgmt,private-network-ip=10.0.3.16,address=35.226.235.236 --metadata-from-file user-data=/home/gcloud/config/slave.conf

To test FortiGate-VM HA:

1. Ensure that the HA status is in-sync and that the public external IP address (104.154.241.0 in this example) is attached to the primary FortiGate:

   FGT-A # get sys ha status

   HA Health Status: OK

   Model: FortiGate-VM64-GCPONDEMAND

   Mode: HA A-P

   Group: 21

   Debug: 0

   Cluster Uptime: 0 days 3:7:1

   Cluster state change time: 2019-01-16 17:17:11

   Master selected using:

   <2019/01/16 17:17:11> FGTGCPA2DHFS8822 is selected as the master because it has the largest value of override priority.

   <2019/01/16 17:17:11> FGTGCPA2DHFS8822 is selected as the master because it's the only member in the cluster.

   ses_pickup: enable, ses_pickup_delay=disable

   override: enable

   unicast_hb: peerip=10.0.2.16, myip=10.0.2.15, hasync_port='port3'

   Configuration Status:

   FGTGCPA2DHFS8822(updated 4 seconds ago): in-sync

   FGTGCPVXW2MYFH07(updated 3 seconds ago): in-sync

   

2. Log in to the PC.
3. Verify that the PC can access the Internet via FortiGate A, since FortiGate A is the primary FortiGate. Verify that the route-internal route gateway is set as 10.0.1.15, the FortiGate A IP address.
4. Shut down FortiGate A.
5. Verify that FortiGate B is now the primary FortiGate.
6. Using an API call, ensure that the route-internal route was removed and replaced with a new one, which has set the gateway as 10.0.1.16, the FortiGate B IP address.

   

7. Verify that the public IP address has detached from FortiGate A and is attached to FortiGate B.
8. Log into the PC.
9. Verify that the PC can access the Internet via FortiGate B, since FortiGate B is now the primary FortiGate.

To configure a VDOM exception:

You must configure a VDOM exception to prevent interface synchronization between the two FortiGates. FortiOS 6.4.1 and later versions support the following commands. FortiOS 6.4.0 does not support these commands.

config system vdom-exception

edit 1

set object system.interface

next

edit 2

set object router.static

next

edit 3

set object firewall.vip

next

end

To run diagnose commands:

After FortiGate A is shut down and FortiGate B becomes the new primary FortiGate, run the following diagnose command to see what happened to the route and public external IP address during the failover procedure:

FGT-B # diagnose debug application gcpd -1

The following shows the procedure of removing the old route (route-internal) and replacing it with a new route:

failover route: route-internal (destRange: 0.0.0.0/0, nextHop: 10.0.1.15)

move next hop from 10.0.1.15 to 10.0.1.16

remove route route-internal on next hop 10.0.1.15

create route route-internal on next hop 10.0.1.16

gcpd api post data: { "name": "route-internal", "network": "https://www.googleapis.com/compute/v1/projects/dev-project-001-166400/global/networks/hapvc-port2internal", "destRange": "0.0.0.0/0", "nextHopIp": "10.0.1.16", "priority": "1000" }

route route-internal is updated to next hop 10.0.1.16 successfully.

The following shows the procedure of attaching a public external IP address to the new primary FortiGate B:

eip: reserve-fgthapublic(104.154.241.0)

eip reserve-fgthapublic(104.154.241.0) is attached in remote instance: us-central1-c/fgt-a, should be moved to local

get instance nic: nic0, 10.0.0.15, hapvc-port1external, accessConfig(external-nat), eip(104.154.241.0)

nic0 of instance fgt-a is using eip 104.154.241.0

remove eip 104.154.241.0 from instance fgt-a(nic0).

attach eip 104.154.241.0 to instance fgt-b(nic0).

gcpd api post data: { "name": "external-nat", "natIP": "104.154.241.0"}

eip reserve-fgthapublic(104.154.241.0) is attached to local successfully.