FGCP over FGSP per-tunnel failover for IPsec

For additional redundancy, an FGCP cluster on one site may form FGSP peering with FGCP clusters on other sites. The FGCP over FGSP peers can still synchronize IPsec SAs and act as the primary gateway for individual tunnels for the same dialup servers. When failover happens within an FGCP cluster, tunnel traffic will failover to the other FGCP cluster member. When an FGCP cluster fails, tunnel traffic will failover to the other FGSP peer.

Example

In this example, each FGCP A-P cluster is connected on port4 as the heartbeat interface. The FGSP peers are connected on port5 over 172.31.2.1-2/24. Each FGSP peer and FGCP cluster has a loopback interface, lb1, with the same IP address. This loopback interface is used as the local gateway on each of the phase 1 connections to avoid each FGSP member having different IPs on port2. The DC Router uses ECMP to distribute traffic to each FGSP peer. It is assumed that the networking addresses are already configured properly.

Interface/setting	DC2_VM1	DC2_VM2	DC2_VM3	DC2_VM4
port2	192.168.129.254/24	192.168.129.254/24	192.168.130.254/24	192.168.130.254/24
port3	172.31.129.254/24	172.31.129.254/24	172.31.130.254/24	172.31.130.254/24
port4	FGCP HA heartbeat interface	FGCP HA heartbeat interface	FGCP HA heartbeat interface	FGCP HA heartbeat interface
port5	172.31.2.1/24	172.31.2.1/24	172.31.2.2/24	172.31.2.2/24
lb1	192.168.202.35/32	192.168.202.35/32	192.168.202.35/32	192.168.205.35/32
fgsp-sync	Enabled	Enabled	Enabled	Enabled

There are two pairs of FGCP A-P HA clusters that form FGSP peering with each other. This is a typical FGCP over FGSP configuration used in large enterprises and service provider environments where high redundancy is needed. Each cluster uses the same loopback address for the local gateway. The DC Router uses ECMP to route traffic to the destination 192.168.202.31 through each of the participating FGSP peers.

In a larger scale there may be many more members in the FGCP clusters, more FGSP peers, and more IPsec dialup clients connecting. Each eligible FGSP peer will be the primary gateway for a set of dialup tunnels, and is in standby for the rest of the tunnels. When the FGCP cluster is configured in A-P mode, the tunnels will be established on the primary unit and synchronized to the standby unit.

The following configurations and example demonstrates PC1 initiating traffic to the Server. First, a dialup tunnel is formed between FortiGate IPsec Client 1 and DC2_VM1, which allows traffic to go through. IPsec SAs are synchronized to the FGCP standby unit, and to the FGSP peer. Upon failure of DC2_VM1, DC2_VM2 takes over as the primary of the HA cluster, and assumes the primary role for the failover tunnels.

If both DC2_VM1 and DC2_VM2 fail, the tunnels that were formed on this FGSP peer will now be re-routed to the other FGSP peer. The primary FGCP cluster member, DC2_VM3, will now pick up the tunnel traffic and assume the primary role for the failover tunnels.

To configure the HA clusters:

1. Configure FGCP A-P Cluster 1 (use the same configuration for DC2_VM1 and DC2_VM2):

   config system ha
       set group-id 1
       set group-name "DC2_VM12"
       set mode a-p
       set password ********
       set hbdev "port4" 50
       set session-pickup enable
       set uninterruptible-upgrade disable
       set override disable
       set priority 100
   end

2. Configure FGCP A-P Cluster 2 (use the same configuration for DC2_VM3 and DC2_VM4):

   config system ha
       set group-id 2
       set group-name "DC2_VM34"
       set mode a-p
       set password ********
       set hbdev "port4" 50
       set session-pickup enable
       set uninterruptible-upgrade disable
       set override disable
       set priority 100
   end

To configure the FGSP peers:

1. Configure DC2_VM1:

   config system standalone-cluster
       set standalone-group-id 2
       set group-member-id 1
       config cluster-peer
           edit 1
               set peerip 172.31.2.2
           next
       end
   end

   The configuration is automatically synchronized to DC2_VM2.

2. Configure DC2_VM3:

   config system standalone-cluster
       set standalone-group-id 2
       set group-member-id 2
       config cluster-peer
           edit 1
               set peerip 172.31.2.1
           next
       end
   end

   The configuration is automatically synchronized to DC2_VM4.

3. To configure the IPsec VPN settings (use the same configuration for DC2_VM1 and DC2_VM3).

   1. Configure the VPN tunnel phase 1 settings:

      config vpn ipsec phase1-interface
          edit "vpn1"
              set type dynamic
              set interface "port2"
              set ike-version 2
              set local-gw 192.168.202.35
              set keylife 90000
              set peertype one
              set net-device disable
              set proposal aes128-sha1
              set add-route disable
              set dpd on-idle
              set dhgrp 2
              set fgsp-sync enable
              set nattraversal disable
              set peerid "Nokia_Peer"
              set psksecret ********
              set dpd-retryinterval 60
          next
      end

   2. Configure the VPN tunnel phase 2 settings:

      config vpn ipsec phase2-interface
          edit "vpn1"
              set phase1name "vpn1"
              set proposal aes128-sha1
              set keylifeseconds 10800
          next
      end

To verify the configuration:

1. The FGCP HA cluster and the FGSP peering have formed. Verify the respective HA statuses.

   1. Verify the FGCP cluster status on DC2_VM1:

      DC2_VM1 # diagnose sys ha status
      
      HA information
      Statistics
              traffic.local = s:0 p:439253 b:89121494
              traffic.total = s:0 p:440309 b:89242174
              activity.ha_id_changes = 2
              activity.fdb  = c:0 q:0
      
      Model=80006, Mode=2 Group=1 Debug=0
      nvcluster=1, ses_pickup=1, delay=0
      
      [Debug_Zone HA information]
      HA group member information: is_manage_primary=1.
      FGVM02TM22000002:      Primary, serialno_prio=0, usr_priority=100, hostname=DC2_VM2
      FGVM02TM22000001:    Secondary, serialno_prio=1, usr_priority=200, hostname=DC2_VM1
      
      [Kernel HA information]
      vcluster 1, state=work, primary_ip=169.254.0.1, primary_id=0
      FGVM02TM22000002:      Primary, ha_prio/o_ha_prio=0/0
      FGVM02TM22000001:    Secondary, ha_prio/o_ha_prio=1/1

   2. Verify the FGSP peering status on DC2_VM1:

      DC2_VM1 # diagnose sys ha standalone-peers
      Group=2, ID=1
      Detected-peers=1
      Kernel standalone-peers: num=1.
      peer0: vfid=0, peerip:port = 172.31.2.2:708, standalone_id=2
              session-type: send=3, recv=4
               packet-type: send=0, recv=0
      Kernel standalone dev_base:
              standalone_id=0:
              standalone_id=1:
                      phyindex=0: mac=00:0c:29:fc:a3:17, linkfail=1
                      phyindex=1: mac=00:0c:29:fc:a3:21, linkfail=1
                      phyindex=2: mac=00:0c:29:fc:a3:2b, linkfail=1
                      phyindex=3: mac=00:0c:29:fc:a3:35, linkfail=1
                      phyindex=4: mac=00:0c:29:fc:a3:3f, linkfail=1
                      phyindex=5: mac=00:0c:29:fc:a3:49, linkfail=1
                      phyindex=6: mac=00:0c:29:fc:a3:53, linkfail=1
                      phyindex=7: mac=00:0c:29:fc:a3:5d, linkfail=1
                      phyindex=8: mac=00:0c:29:fc:a3:67, linkfail=1
                      phyindex=9: mac=00:0c:29:fc:a3:71, linkfail=1
              standalone_id=2:
                      phyindex=0: mac=00:09:0f:09:02:00, linkfail=1
                      phyindex=1: mac=00:09:0f:09:02:01, linkfail=1
                      phyindex=2: mac=00:09:0f:09:02:02, linkfail=1
                      phyindex=3: mac=00:09:0f:09:02:03, linkfail=1
                      phyindex=4: mac=00:09:0f:09:02:04, linkfail=1
                      phyindex=5: mac=00:09:0f:09:02:05, linkfail=1
                      phyindex=6: mac=00:09:0f:09:02:06, linkfail=1
                      phyindex=7: mac=00:09:0f:09:02:07, linkfail=1
                      phyindex=8: mac=00:09:0f:09:02:08, linkfail=1
                      phyindex=9: mac=00:09:0f:09:02:09, linkfail=1
              standalone_id=3:
              ...
              standalone_id=15:

   3. Verify the FGCP cluster status on DC2_VM3:

      DC2_VM3 # diagnose sys ha status
      HA information
      Statistics
              traffic.local = s:0 p:443999 b:89037989
              traffic.total = s:0 p:445048 b:89157373
              activity.ha_id_changes = 2
              activity.fdb  = c:0 q:0
      
      Model=80006, Mode=2 Group=2 Debug=0
      nvcluster=1, ses_pickup=1, delay=0
      
      [Debug_Zone HA information]
      HA group member information: is_manage_primary=1.
      FGVM02TM22000004:      Primary, serialno_prio=0, usr_priority=100, hostname=DC2_VM4
      FGVM02TM22000003:    Secondary, serialno_prio=1, usr_priority=200, hostname=DC2_VM3
      
      [Kernel HA information]
      vcluster 1, state=work, primary_ip=169.254.0.1, primary_id=0
      FGVM02TM22000004:      Primary, ha_prio/o_ha_prio=0/0
      FGVM02TM22000003:    Secondary, ha_prio/o_ha_prio=1/1

   4. Verify the FGSP peering status on DC2_VM3:

      DC2_VM3 # diagnose sys ha standalone-peers
      Group=2, ID=2
      Detected-peers=1
      Kernel standalone-peers: num=1.
      peer0: vfid=0, peerip:port = 172.31.2.1:708, standalone_id=1
              session-type: send=2, recv=6
               packet-type: send=0, recv=0
      Kernel standalone dev_base:
              standalone_id=0:
              standalone_id=1:
                      phyindex=0: mac=00:09:0f:09:01:00, linkfail=1
                      phyindex=1: mac=00:09:0f:09:01:01, linkfail=1
                      phyindex=2: mac=00:09:0f:09:01:02, linkfail=1
                      phyindex=3: mac=00:09:0f:09:01:03, linkfail=1
                      phyindex=4: mac=00:09:0f:09:01:04, linkfail=1
                      phyindex=5: mac=00:09:0f:09:01:05, linkfail=1
                      phyindex=6: mac=00:09:0f:09:01:06, linkfail=1
                      phyindex=7: mac=00:09:0f:09:01:07, linkfail=1
                      phyindex=8: mac=00:09:0f:09:01:08, linkfail=1
                      phyindex=9: mac=00:09:0f:09:01:09, linkfail=1
              standalone_id=2:
                      phyindex=0: mac=00:0c:29:bb:77:af, linkfail=1
                      phyindex=1: mac=00:0c:29:bb:77:b9, linkfail=1
                      phyindex=2: mac=00:0c:29:bb:77:c3, linkfail=1
                      phyindex=3: mac=00:0c:29:bb:77:cd, linkfail=1
                      phyindex=4: mac=00:0c:29:bb:77:d7, linkfail=1
                      phyindex=5: mac=00:0c:29:bb:77:e1, linkfail=1
                      phyindex=6: mac=00:0c:29:bb:77:eb, linkfail=1
                      phyindex=7: mac=00:0c:29:bb:77:f5, linkfail=1
                      phyindex=8: mac=00:0c:29:bb:77:ff, linkfail=1
                      phyindex=9: mac=00:0c:29:bb:77:09, linkfail=1
              standalone_id=3:
              ...
              standalone_id=15:

2. Initiate traffic from PC1 to the Server. This initiates a tunnel from the IPsec Client 1 FortiGate to DC2_VM1.

3. Verify the tunnel list for vpn1_1 on each peer.

   1. DC2_VM1:

      DC2_VM1 # diagnose vpn tunnel list
      list all ipsec tunnel in vd 0
      ------------------------------------------------------
      name=vpn1_1 ver=2 serial=4 192.168.202.35:0->192.168.7.2:0 tun_id=192.168.7.2 tun_id6=::10.0.0.4 dst_mtu=1500 dpd-link=on weight=1
      bound_if=6 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/8840 options[2288]=npu rgwy-chg frag-rfc  run_state=0 role=sync-primary accept_traffic=1 overlay_id=0
      
      parent=vpn1 index=1
      proxyid_num=1 child_num=0 refcnt=5 ilast=41 olast=41 ad=/0
      stat: rxp=0 txp=0 rxb=0 txb=0
      dpd: mode=on-idle on=1 idle=60000ms retry=3 count=0 seqno=156
      natt: mode=none draft=0 interval=0 remote_port=0
      fec: egress=0 ingress=0
      proxyid=vpn1 proto=0 sa=1 ref=2 serial=1
        src: 0:0.0.0.0-255.255.255.255:0
        dst: 0:10.10.1.0-10.10.1.255:0
        SA:  ref=3 options=602 type=00 soft=0 mtu=1438 expire=1424/0B replaywin=2048
             seqno=1 esn=0 replaywin_lastseq=00000000 qat=0 rekey=0 hash_search_len=1
        life: type=01 bytes=0/0 timeout=10791/10800
        dec: spi=37f426a1 esp=aes key=16 3671c9303b6295fc73b11765811bdf96
             ah=sha1 key=20 41b98cb541dc9c76311ddec4b23584ee35d31915
        enc: spi=10aa4d3a esp=aes key=16 cc8529ee16de6e4ac42b0ce506d7cdd1
             ah=sha1 key=20 0c2d9edd0fdbe45942cf718ac2ebb4d59c2760c6
        dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
        npu_flag=00 npu_rgwy=192.168.7.2 npu_lgwy=192.168.202.35 npu_selid=1c dec_npuid=0 enc_npuid=0

   2. DC2_VM2:

      DC2_VM2 # diagnose vpn tunnel list
      list all ipsec tunnel in vd 0
      ------------------------------------------------------
      name=vpn1_1 ver=2 serial=4 192.168.202.35:0->192.168.7.2:0 tun_id=192.168.7.2 tun_id6=::10.0.0.4 dst_mtu=0 dpd-link=on weight=1
      bound_if=6 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/8712 options[2208]=npu frag-rfc  run_state=0 role=standby accept_traffic=1 overlay_id=0
      
      parent=vpn1 index=1
      proxyid_num=1 child_num=0 refcnt=5 ilast=42975898 olast=42975898 ad=/0
      stat: rxp=0 txp=0 rxb=0 txb=0
      dpd: mode=on-idle on=1 idle=60000ms retry=3 count=0 seqno=0
      natt: mode=none draft=0 interval=0 remote_port=0
      fec: egress=0 ingress=0
      proxyid=vpn1 proto=0 sa=1 ref=2 serial=1
        src: 0:0.0.0.0-255.255.255.255:0
        dst: 0:10.10.1.0-10.10.1.255:0
        SA:  ref=3 options=602 type=00 soft=0 mtu=1280 expire=1325/0B replaywin=2048
             seqno=10000001 esn=0 replaywin_lastseq=00000000 qat=0 rekey=0 hash_search_len=1
        life: type=01 bytes=0/0 timeout=10791/10800
        dec: spi=37f426a1 esp=aes key=16 3671c9303b6295fc73b11765811bdf96
             ah=sha1 key=20 41b98cb541dc9c76311ddec4b23584ee35d31915
        enc: spi=10aa4d3a esp=aes key=16 cc8529ee16de6e4ac42b0ce506d7cdd1
             ah=sha1 key=20 0c2d9edd0fdbe45942cf718ac2ebb4d59c2760c6
        dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
        npu_flag=00 npu_rgwy=192.168.7.2 npu_lgwy=192.168.202.35 npu_selid=1c dec_npuid=0 enc_npuid=0

   3. DC2_VM3:

      DC2_VM3 # diagnose vpn tunnel list
      list all ipsec tunnel in vd 0
      ------------------------------------------------------
      name=vpn1_1 ver=2 serial=4 192.168.202.35:0->192.168.7.2:0 tun_id=192.168.7.2 tun_id6=::10.0.0.4 dst_mtu=0 dpd-link=on weight=1
      bound_if=6 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/8712 options[2208]=npu frag-rfc  run_state=0 role=standby accept_traffic=1 overlay_id=0
      
      parent=vpn1 index=1
      proxyid_num=1 child_num=0 refcnt=5 ilast=42975982 olast=42975982 ad=/0
      stat: rxp=0 txp=0 rxb=0 txb=0
      dpd: mode=on-idle on=1 idle=60000ms retry=3 count=0 seqno=0
      natt: mode=none draft=0 interval=0 remote_port=0
      fec: egress=0 ingress=0
      proxyid=vpn1 proto=0 sa=1 ref=2 serial=1
        src: 0:0.0.0.0-255.255.255.255:0
        dst: 0:10.10.1.0-10.10.1.255:0
        SA:  ref=3 options=602 type=00 soft=0 mtu=1280 expire=1215/0B replaywin=2048
             seqno=10000001 esn=0 replaywin_lastseq=00000000 qat=0 rekey=0 hash_search_len=1
        life: type=01 bytes=0/0 timeout=10791/10800
        dec: spi=37f426a1 esp=aes key=16 3671c9303b6295fc73b11765811bdf96
             ah=sha1 key=20 41b98cb541dc9c76311ddec4b23584ee35d31915
        enc: spi=10aa4d3a esp=aes key=16 cc8529ee16de6e4ac42b0ce506d7cdd1
             ah=sha1 key=20 0c2d9edd0fdbe45942cf718ac2ebb4d59c2760c6
        dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
        npu_flag=00 npu_rgwy=192.168.7.2 npu_lgwy=192.168.202.35 npu_selid=1c dec_npuid=0 enc_npuid=0

   4. DC2_VM4:

      DC2_VM4 # diagnose vpn  tunnel list
      list all ipsec tunnel in vd 0
      ------------------------------------------------------
      name=vpn1_1 ver=2 serial=4 192.168.202.35:0->192.168.7.2:0 tun_id=192.168.7.2 tun_id6=::10.0.0.4 dst_mtu=0 dpd-link=on weight=1
      bound_if=6 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/8712 options[2208]=npu frag-rfc  run_state=0 role=standby accept_traffic=1 overlay_id=0
      
      parent=vpn1 index=1
      proxyid_num=1 child_num=0 refcnt=5 ilast=42975768 olast=42975768 ad=/0
      stat: rxp=0 txp=0 rxb=0 txb=0
      dpd: mode=on-idle on=1 idle=60000ms retry=3 count=0 seqno=0
      natt: mode=none draft=0 interval=0 remote_port=0
      fec: egress=0 ingress=0
      proxyid=vpn1 proto=0 sa=1 ref=2 serial=1
        src: 0:0.0.0.0-255.255.255.255:0
        dst: 0:10.10.1.0-10.10.1.255:0
        SA:  ref=3 options=602 type=00 soft=0 mtu=1280 expire=1433/0B replaywin=2048
             seqno=10000001 esn=0 replaywin_lastseq=00000000 qat=0 rekey=0 hash_search_len=1
        life: type=01 bytes=0/0 timeout=10791/10800
        dec: spi=37f426a1 esp=aes key=16 3671c9303b6295fc73b11765811bdf96
             ah=sha1 key=20 41b98cb541dc9c76311ddec4b23584ee35d31915
        enc: spi=10aa4d3a esp=aes key=16 cc8529ee16de6e4ac42b0ce506d7cdd1
             ah=sha1 key=20 0c2d9edd0fdbe45942cf718ac2ebb4d59c2760c6
        dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
        npu_flag=00 npu_rgwy=192.168.7.2 npu_lgwy=192.168.202.35 npu_selid=1c dec_npuid=0 enc_npuid=0

      The IPsec tunnel role=sync-primaryon DC2_VM1 indicates that it is being used to carry IPsec traffic. On DC2_VM2, DC2_VM3, and DC2_VM4, the IPsec tunnel role=standby indicates that they are in standby for traffic forwarding.

To test failover scenarios:

1. Verify the sniffer trace on DC2_VM1 before FGCP HA failover:

   DC2_VM1 # diagnose sniffer packet any icmp 4
   Using Original Sniffing Mode
   interfaces=[any]
   filters=[icmp]
   0.171753 vpn1 in 10.10.1.2 -> 10.10.101.2: icmp: echo request
   0.171763 port3 out 10.10.1.2 -> 10.10.101.2: icmp: echo request
   0.171941 port3 in 10.10.101.2 -> 10.10.1.2: icmp: echo reply
   0.171947 vpn1 out 10.10.101.2 -> 10.10.1.2: icmp: echo reply

   Traffic passes through DC2_VM1.

2. Reboot the primary FortiGate, DC2_VM1.

3. Verify the sniffer trace on DC2_VM2 after FGCP HA failover:

   DC2_VM2 # diagnose sniffer packet any icmp 4
   Using Original Sniffing Mode
   interfaces=[any]
   filters=[icmp]
   0.111107 vpn1 in 10.10.1.2 -> 10.10.101.2: icmp: echo request
   0.111118 port3 out 10.10.1.2 -> 10.10.101.2: icmp: echo request
   0.111293 port3 in 10.10.101.2 -> 10.10.1.2: icmp: echo reply
   0.111298 vpn1 out 10.10.101.2 -> 10.10.1.2: icmp: echo reply
   ^C
   16 packets received by filter
   0 packets dropped by kernel

   Traffic passes through DC2_VM2.

4. Verify the tunnel list for vpn1_1 on DC2_VM2:

   DC2_VM2 # diagnose vpn tunnel list
   list all ipsec tunnel in vd 0
   ------------------------------------------------------
   name=vpn1_1 ver=2 serial=4 192.168.202.35:0->192.168.7.2:0 tun_id=192.168.7.2 tun_id6=::10.0.0.4 dst_mtu=1500 dpd-link=on weight=1
   bound_if=6 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/8840 options[2288]=npu rgwy-chg frag-rfc  run_state=0 role=sync-primary accept_traffic=1 overlay_id=0
   
   parent=vpn1 index=1
   proxyid_num=1 child_num=0 refcnt=5 ilast=0 olast=0 ad=/0
   stat: rxp=58 txp=31 rxb=4872 txb=2604
   dpd: mode=on-idle on=1 idle=60000ms retry=3 count=0 seqno=169
   natt: mode=none draft=0 interval=0 remote_port=0
   fec: egress=0 ingress=0
   proxyid=vpn1 proto=0 sa=1 ref=3 serial=3
     src: 0:0.0.0.0-255.255.255.255:0
     dst: 0:10.10.1.0-10.10.1.255:0
     SA:  ref=3 options=602 type=00 soft=0 mtu=1438 expire=10730/0B replaywin=2048
          seqno=20 esn=0 replaywin_lastseq=0000003b qat=0 rekey=0 hash_search_len=1
     life: type=01 bytes=0/0 timeout=10790/10800
     dec: spi=37f426c1 esp=aes key=16 ef61b49078b6ab3e00a4d3a048d779f5
          ah=sha1 key=20 ee2e8de9c522d89b6481c37faa73a7bb54163645
     enc: spi=10aa4d58 esp=aes key=16 4cb95f12657ca8e269b9f8a25f9b19c1
          ah=sha1 key=20 326744c4e5b4a0758397725464593d94ba9390dc
     dec:pkts/bytes=116/9744, enc:pkts/bytes=62/7316
     npu_flag=00 npu_rgwy=192.168.7.2 npu_lgwy=192.168.202.35 npu_selid=1e dec_npuid=0 enc_npuid=0

   The role has changed to role=sync-primary.

5. Shut down DC2_VM1 and the DC2_VM2 IPsec uplink interface.

6. Verify the sniffer trace on DC2_VM3. As expected, traffic now passes through DC2_VM3:

   DC2_VM3 # diagnose sniffer packet any icmp 4
   Using Original Sniffing Mode
   interfaces=[any]
   filters=[icmp]
   0.165088 vpn1 in 10.10.1.2 -> 10.10.101.2: icmp: echo request
   0.165102 port3 out 10.10.1.2 -> 10.10.101.2: icmp: echo request
   0.165294 port3 in 10.10.101.2 -> 10.10.1.2: icmp: echo reply
   0.165301 vpn1 out 10.10.101.2 -> 10.10.1.2: icmp: echo reply
   ^C
   14 packets received by filter
   0 packets dropped by kernel

7. Verify the tunnel list for vpn1_1 on DC2_VM3:

   DC2_VM3 # diagnose vpn tunnel list
   list all ipsec tunnel in vd 0
   ------------------------------------------------------
   name=vpn1_1 ver=2 serial=4 192.168.202.35:0->192.168.7.2:0 tun_id=192.168.7.2 tun_id6=::10.0.0.4 dst_mtu=1500 dpd-link=on weight=1
   bound_if=6 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/8712 options[2208]=npu frag-rfc  run_state=0 role=sync-primary accept_traffic=1 overlay_id=0
   
   parent=vpn1 index=1
   proxyid_num=1 child_num=0 refcnt=5 ilast=0 olast=0 ad=/0
   stat: rxp=53 txp=53 rxb=4452 txb=4452
   dpd: mode=on-idle on=1 idle=60000ms retry=3 count=0 seqno=0
   natt: mode=none draft=0 interval=0 remote_port=0
   fec: egress=0 ingress=0
   proxyid=vpn1 proto=0 sa=1 ref=3 serial=3
     src: 0:0.0.0.0-255.255.255.255:0
     dst: 0:10.10.1.0-10.10.1.255:0
     SA:  ref=3 options=602 type=00 soft=0 mtu=1438 expire=10347/0B replaywin=2048
          seqno=10000155 esn=0 replaywin_lastseq=000001b0 qat=0 rekey=0 hash_search_len=1
     life: type=01 bytes=0/0 timeout=10790/10800
     dec: spi=37f426c1 esp=aes key=16 ef61b49078b6ab3e00a4d3a048d779f5
          ah=sha1 key=20 ee2e8de9c522d89b6481c37faa73a7bb54163645
     enc: spi=10aa4d58 esp=aes key=16 4cb95f12657ca8e269b9f8a25f9b19c1
          ah=sha1 key=20 326744c4e5b4a0758397725464593d94ba9390dc
     dec:pkts/bytes=88/7392, enc:pkts/bytes=88/10384
     npu_flag=00 npu_rgwy=192.168.7.2 npu_lgwy=192.168.202.35 npu_selid=1e dec_npuid=0 enc_npuid=0

   The role has changed to role=sync-primary.