Fortinet black logo

Cookbook

Troubleshooting OCVPN

Copy Link
Copy Doc ID 664e9f16-22ad-11eb-96b9-00505692583a:34927
Download PDF

Troubleshooting OCVPN

This document includes troubleshooting steps for the following OCVPN network topologies:

  • Full mesh OCVPN.
  • Hub-spoke OCVPN with ADVPN shortcut.
  • Hub-spoke OCVPN with inter-overlay source NAT.

For OCVPN configurations in other network topologies, see the other OCVPN topics.

Troubleshooting full mesh network topology

  • Branch_1 # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Full-Mesh
    Role                 : Spoke
    Server Status        : Up
    Registration time    : Thu Feb 28 18:42:25 2019
    Update time          : Thu Feb 28 15:57:18 2019
    Poll time            : Fri Mar  1 15:02:28 2019
  • Branch_1 # diagnose vpn ocvpn show-meta
    Topology :: auto
    License  :: full
    Members  :: 3
    Max-free :: 3
  • Branch_1 # diagnose vpn ocvpn show-overlays
    QA
    PM
  • Branch_1 # diagnose vpn ocvpn show-members
    Member: { "SN": "FG100D3G15801621", "IPv4": "172.16.200.1", "port": "500", "slot": 1000, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "10.1.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "10.2.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "Name": "FortiGate-100D", "topology_role": "spoke" } 
    Member: { "SN": "FG900D3915800083", "IPv4": "172.16.200.4", "port": "500", "slot": 1001, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "Name": "Branch3", "topology_role": "spoke" } 
    Member: { "SN": "FGT51E3U16001314", "IPv4": "172.16.200.199", "port": "500", "slot": 1002, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "192.168.4.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "192.168.5.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "Name": "Branch2", "topology_role": "spoke" }  
  • Branch_1 # diagnose vpn tunnel list
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-3.1 ver=2 serial=4 172.16.200.1:0->172.16.200.199:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=13 ilast=7 olast=0 ad=/0
    stat: rxp=0 txp=7 rxb=0 txb=588
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=6
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-3.1 proto=0 sa=1 ref=2 serial=8 auto-negotiate
      src: 0:10.1.100.0-10.1.100.255:0
      dst: 0:192.168.4.0-192.168.4.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42923/0B replaywin=2048
           seqno=8 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42931/43200
      dec: spi=c34bb752 esp=aes key=16 3c5ceeff3cac1eaa2702b5ccb713ab9b
           ah=sha1 key=20 5903e358b3d8938ee64f0412887a0fe741ccb105
      enc: spi=b5bd4fe1 esp=aes key=16 8ae97a8abe24dae725d614d2a6efdcb0
           ah=sha1 key=20 9ec200d9c0cef9e1b7cf76e05dbf344c70f53214
      dec:pkts/bytes=0/0, enc:pkts/bytes=7/1064
    proxyid=_OCVPN2-3.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-4.1 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=11 ilast=19 olast=19 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-4.1 proto=0 sa=1 ref=2 serial=7 auto-negotiate
      src: 0:10.1.100.0-10.1.100.255:0
      dst: 0:172.16.101.0-172.16.101.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42911/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42931/43200
      dec: spi=c34bb750 esp=aes key=16 8c9844a8bcd3fda6c7bd8a4f2ec81ef1
           ah=sha1 key=20 680c7144346f5b52126cbad9f325821b048c7192
      enc: spi=f2d1f2d4 esp=aes key=16 f9625fc8590152829eb39eecab3a3999
           ah=sha1 key=20 5df8447416da541fa54dde9fa3e5c35fbfc4723f
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-4.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-3.2 ver=2 serial=3 172.16.200.1:0->172.16.200.199:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=11 ilast=6 olast=6 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-3.2 proto=0 sa=1 ref=2 serial=8 auto-negotiate
      src: 0:10.2.100.0-10.2.100.255:0
      dst: 0:192.168.5.0-192.168.5.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42923/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42930/43200
      dec: spi=c34bb753 esp=aes key=16 58ddfad9a3699f1c49f3a9f369145c28
           ah=sha1 key=20 e749c7e6a7aaff119707c792eb73cd975127873b
      enc: spi=b5bd4fe2 esp=aes key=16 8f2366e653f5f9ad6587be1ce1905764
           ah=sha1 key=20 5347bf24e51219d483c0f7b058eceab202026204
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-3.2 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-4.2 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=11 ilast=17 olast=17 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-4.2 proto=0 sa=1 ref=2 serial=7 auto-negotiate
      src: 0:10.2.100.0-10.2.100.255:0
      dst: 0:172.16.102.0-172.16.102.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42905/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42927/43200
      dec: spi=c34bb751 esp=aes key=16 41449ee5ea43d3e1f80df05fc632cd44
           ah=sha1 key=20 3ca2aea1c8764f35ccf987cdeca7cf6eb54331fb
      enc: spi=f2d1f2d5 esp=aes key=16 9010dd57e502c6296b27a4649a45a6ba
           ah=sha1 key=20 caf86a176ce04464221543f15fc3c63fc573b8ee
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-4.2 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
  • Branch_1 # get router info routing-table all
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-3.2
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-4.1
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-4.2
    S       192.168.4.0/24 [20/0] is directly connected, _OCVPN2-3.1

Troubleshooting hub-spoke with ADVPN shortcut

  • Primary-Hub # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Primary-Hub
    Server Status        : Up
    Registration time    : Sat Mar  2 11:31:54 2019
    Poll time            : Sat Mar  2 11:46:02 2019
  • Spoke1 # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Spoke
    Server Status        : Up
    Registration time    : Sat Mar  2 11:41:22 2019
    Poll time            : Sat Mar  2 11:46:44 2019
  • Primary-Hub # diagnose vpn ocvpn show-members
    Member: { "sn": "FG900D3915800083", "ip_v4": "172.16.200.4", "port": 500, "slot": 0, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Primary-Hub", "topology_role": "primary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FG100D3G15828488", "ip_v4": "172.16.200.2", "port": 500, "slot": 1, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Secondary-Hub", "topology_role": "secondary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FG100D3G15801621", "ip_v4": "172.16.200.1", "port": 500, "slot": 1000, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "10.1.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "10.2.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke1", "topology_role": "spoke" }
    Member: { "sn": "FGT51E3U16001314", "ip_v4": "172.16.200.3", "port": 500, "slot": 1001, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "192.168.4.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "192.168.5.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke2", "topology_role": "spoke" }
  • Primary-Hub # diagnose vpn ocvpn show-meta
    Topology :: auto
    License  :: full
    Members  :: 4
    Max-free :: 3
  • Primary-Hub # diagnose vpn ocvpn show-overlays
    QA
    PM
  • Spoke1 # diagnose vpn tunnel list
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=0 olast=0 ad=r/2
    stat: rxp=1 txp=34 rxb=152 txb=2856
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=46
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42895/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c7 esp=aes key=16 240e064c0f1c980ca31980b9e7605c9d
           ah=sha1 key=20 6ff022cbebcaff4c5de62eefb2e6180c40a3adb2
      enc: spi=dfcffa86 esp=aes key=16 862208de164a02af377756c2bcabd588
           ah=sha1 key=20 af6e54781fd42d7a2ba2119ec95d0f95629c8448
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=8 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=934 olast=934 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=12 olast=12 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=46
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42895/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c8 esp=aes key=16 701ec608767f4988b76c2f662464e654
           ah=sha1 key=20 93c65d106dc610d7ee3f04487f08601a9e00ffdd
      enc: spi=dfcffa87 esp=aes key=16 02b2d04dce3d81ebab69e128d45cb7ca
           ah=sha1 key=20 4a9283847f852c83a75691fad44d07d8409a2267
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=7 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=934 olast=934 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
  • Spoke1 # get router info routing-table all
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-0.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.4.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-0.1
  • Generate traffic from spoke1 to spoke2 to trigger the ADVPN shortcut and check the VPN tunnel and routing-table again on spoke1.
    branch1 # diagnose vpn tunnel list 
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0_0 ver=2 serial=a 172.16.200.1:0->172.16.200.3:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/720 options[02d0]=create_dev no-sysctl rgwy-chg frag-rfc  accept_traffic=1
    
     parent=_OCVPN2-0.0 index=0
    proxyid_num=1 child_num=0 refcnt=14 ilast=0 olast=0 ad=r/2
    stat: rxp=7 txp=7 rxb=1064 txb=588
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate add-route adr
      src: 0:10.1.100.0-10.1.100.255:0
      dst: 0:192.168.4.0-192.168.4.255:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=43180/0B replaywin=2048
           seqno=8 esn=0 replaywin_lastseq=00000008 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=43187/43200
      dec: spi=048477c9 esp=aes key=16 27c35d53793013ef24cf887561e9f313
           ah=sha1 key=20 2c8cfd328c3b29104db0ca74a00c6063f46cafe4
      enc: spi=fb9e13fd esp=aes key=16 9d0d3bf6c84b7ddaf9d9196fe74002ed
           ah=sha1 key=20 d1f541db787dea384c6a4df16fc228abeb7ae334
      dec:pkts/bytes=7/588, enc:pkts/bytes=7/1064
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=1 refcnt=12 ilast=7 olast=7 ad=r/2
    stat: rxp=2 txp=35 rxb=304 txb=2940
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=65
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42500/0B replaywin=2048
           seqno=2 esn=0 replaywin_lastseq=00000002 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c7 esp=aes key=16 240e064c0f1c980ca31980b9e7605c9d
           ah=sha1 key=20 6ff022cbebcaff4c5de62eefb2e6180c40a3adb2
      enc: spi=dfcffa86 esp=aes key=16 862208de164a02af377756c2bcabd588
           ah=sha1 key=20 af6e54781fd42d7a2ba2119ec95d0f95629c8448
      dec:pkts/bytes=1/84, enc:pkts/bytes=1/152
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=8 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=1328 olast=1328 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=5 olast=5 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=66
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42500/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c8 esp=aes key=16 701ec608767f4988b76c2f662464e654
           ah=sha1 key=20 93c65d106dc610d7ee3f04487f08601a9e00ffdd
      enc: spi=dfcffa87 esp=aes key=16 02b2d04dce3d81ebab69e128d45cb7ca
           ah=sha1 key=20 4a9283847f852c83a75691fad44d07d8409a2267
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=7 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=1328 olast=1328 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    
    
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-0.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.4.0/24 [15/0] via 172.16.200.3, _OCVPN2-0.0_0
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-0.1
  • Simulate the primary hub being unavailable where all spokes' dialup VPN tunnels will switch to the secondary hub, to check VPN tunnel status and routing-table.
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=25 olast=25 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=82
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=8 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=14 olast=14 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=9
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42723/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42898/43200
      dec: spi=048477cd esp=aes key=16 9bb363a32378b5897cd42890c92df811
           ah=sha1 key=20 2ed40583b9544e37867349b4adc7c013024d7e17
      enc: spi=f345fb42 esp=aes key=16 3ea31dff3310b245700a131db4565851
           ah=sha1 key=20 522862dfb232514b845e436133b148da0e67b7c4
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=19 olast=19 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=83
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=7 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=12 olast=12 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=9
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42728/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42902/43200
      dec: spi=048477cf esp=aes key=16 b6f0ca7564abcd8559b5b0ebb3fd04c1
           ah=sha1 key=20 4130d040554b39daca72adac7583b9cc83cce3c8
      enc: spi=f345fb43 esp=aes key=16 727582f20fcedff884ba693ed2164bcd
           ah=sha1 key=20 b0a625803fde701ed9d28d256079e908954b7fc8
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    
    
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.102.0/24 [21/0] is directly connected, _OCVPN2-1.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [21/0] is directly connected, _OCVPN2-1.0
    S       192.168.4.0/24 [21/0] is directly connected, _OCVPN2-1.0
    S       192.168.5.0/24 [21/0] is directly connected, _OCVPN2-1.1

Troubleshooting hub-spoke with inter-overlay source NAT

  • Primary-Hub # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Primary-Hub
    Server Status        : Up
    Registration time    : Sat Mar  2 11:31:54 2019
    Update time          : Sat Mar  2 13:57:05 2019
    Poll time            : Sat Mar  2 14:03:31 2019
  • Spoke1 # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Spoke
    Server Status        : Up
    Registration time    : Sat Mar  2 13:58:01 2019
    Poll time            : Sat Mar  2 14:04:22 2019
  • Primary-Hub # diagnose vpn ocvpn show-members
    Member: { "sn": "FG900D3915800083", "ip_v4": "172.16.200.4", "port": 500, "slot": 0, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "172.16.101.100-172.16.101.200" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "172.16.102.100-172.16.102.200" } ], "name": "Primary-Hub", "topology_role": "primary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FG100D3G15828488", "ip_v4": "172.16.200.2", "port": 500, "slot": 1, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Secondary-Hub", "topology_role": "secondary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FGT51E3U16001314", "ip_v4": "172.16.200.3", "port": 500, "slot": 1001, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "192.168.4.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "192.168.5.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke2", "topology_role": "spoke" }
    Member: { "sn": "FG100D3G15801621", "ip_v4": "172.16.200.1", "port": 500, "slot": 1000, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "10.1.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "10.2.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke1", "topology_role": "spoke" }
  • Primary-Hub # diagnose vpn ocvpn show-meta
    Topology :: auto
    License  :: full
    Members  :: 4
    Max-free :: 3
  • Primary-Hub # diagnose vpn ocvpn show-overlays
    QA
    PM
  • Spoke1 # diagnose vpn tunnel list
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=c 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=3 child_num=0 refcnt=13 ilast=17 olast=17 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=29
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=18227 type=00 soft=0 mtu=1438 expire=42299/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42899/43200
      dec: spi=0484795d esp=aes key=16 10eeb76fadd49f00c333350d83509095
           ah=sha1 key=20 971bde5dcfca7e52fd1573cb3489e9c855f6154e
      enc: spi=dfcffaaa esp=aes key=16 d07a4dd683ee093af2dca9485aa436eb
           ah=sha1 key=20 65369be35d5ecad8cae63557318419cd6005c230
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.0_nat proto=0 sa=1 ref=2 serial=3 auto-negotiate
      src: 0:172.16.101.101-172.16.101.101:0
      dst: 0:0.0.0.0-255.255.255.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42303/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42898/43200
      dec: spi=04847961 esp=aes key=16 ea181036b02e8bc8711fb520b3e98a60
           ah=sha1 key=20 b3c449d96d5d3f090975087a62447f6918ce7930
      enc: spi=dfcffaac esp=aes key=16 f7ea5e42e9443698e6b8b32161ace40e
           ah=sha1 key=20 a7e36dd1ec0bdb6eff0aa66e442707427400c700
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.0_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=e 172.16.200.1:0->172.16.200.2:0 dst_mtu=0
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=10 ilast=599 olast=599 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    proxyid=_OCVPN2-1.0_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=b 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=3 child_num=0 refcnt=13 ilast=17 olast=17 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=29
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=18227 type=00 soft=0 mtu=1438 expire=42297/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42897/43200
      dec: spi=0484795e esp=aes key=16 106eaa95a2be64b566e7d1ca0aa88f6a
           ah=sha1 key=20 5dddfba7070b03d5a31931d41db06ff96e7bc542
      enc: spi=dfcffaab esp=aes key=16 29c774dbd7e54464ee298c381e71a94e
           ah=sha1 key=20 c3da7372789c0a53b3752e69baaba1a42d798820
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.1_nat proto=0 sa=1 ref=2 serial=3 auto-negotiate
      src: 0:172.16.102.101-172.16.102.101:0
      dst: 0:0.0.0.0-255.255.255.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42307/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42902/43200
      dec: spi=04847962 esp=aes key=16 b7daa5807cfa86906592a012a9d2478f
           ah=sha1 key=20 39c8bb4c9e3f1e9e451f22c58a172ff01155055d
      enc: spi=dfcffaad esp=aes key=16 2ecc644def4cebe6b0c4b7729da43d8e
           ah=sha1 key=20 469c6f319e83bd73468f55d430566afcd6215138
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.1_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=d 172.16.200.1:0->172.16.200.2:0 dst_mtu=0
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=10 ilast=599 olast=599 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    proxyid=_OCVPN2-1.1_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
  • Spoke1 # get router info routing-table all
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-0.1
    C       172.16.101.101/32 is directly connected, _OCVPN2-0.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-0.0
    C       172.16.102.101/32 is directly connected, _OCVPN2-0.0
    S       192.168.4.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-0.1
  • Spoke1 # show firewall policy
     ..............................
    
        edit 9
            set name "_OCVPN2-1.1_nat"
            set uuid 3f7a84b8-3d36-51e9-ee97-8f418c91e666
            set srcintf "any"
            set dstintf "_OCVPN2-1.1"
            set srcaddr "all"
            set dstaddr "_OCVPN2-1.1_remote_networks"
            set action accept
            set schedule "always"
            set service "ALL"
            set comments "Generated by OCVPN Cloud Service."
            set nat enable
        next
        edit 12
            set name "_OCVPN2-1.0_nat"
            set uuid 3fafec98-3d36-51e9-80c0-5d99325bad83
            set srcintf "any"
            set dstintf "_OCVPN2-1.0"
            set srcaddr "all"
            set dstaddr "_OCVPN2-1.0_remote_networks"
            set action accept
            set schedule "always"
            set service "ALL"
            set comments "Generated by OCVPN Cloud Service."
            set nat enable
        next
       .................................

Troubleshooting OCVPN

This document includes troubleshooting steps for the following OCVPN network topologies:

  • Full mesh OCVPN.
  • Hub-spoke OCVPN with ADVPN shortcut.
  • Hub-spoke OCVPN with inter-overlay source NAT.

For OCVPN configurations in other network topologies, see the other OCVPN topics.

Troubleshooting full mesh network topology

  • Branch_1 # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Full-Mesh
    Role                 : Spoke
    Server Status        : Up
    Registration time    : Thu Feb 28 18:42:25 2019
    Update time          : Thu Feb 28 15:57:18 2019
    Poll time            : Fri Mar  1 15:02:28 2019
  • Branch_1 # diagnose vpn ocvpn show-meta
    Topology :: auto
    License  :: full
    Members  :: 3
    Max-free :: 3
  • Branch_1 # diagnose vpn ocvpn show-overlays
    QA
    PM
  • Branch_1 # diagnose vpn ocvpn show-members
    Member: { "SN": "FG100D3G15801621", "IPv4": "172.16.200.1", "port": "500", "slot": 1000, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "10.1.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "10.2.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "Name": "FortiGate-100D", "topology_role": "spoke" } 
    Member: { "SN": "FG900D3915800083", "IPv4": "172.16.200.4", "port": "500", "slot": 1001, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "Name": "Branch3", "topology_role": "spoke" } 
    Member: { "SN": "FGT51E3U16001314", "IPv4": "172.16.200.199", "port": "500", "slot": 1002, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "192.168.4.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "192.168.5.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "Name": "Branch2", "topology_role": "spoke" }  
  • Branch_1 # diagnose vpn tunnel list
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-3.1 ver=2 serial=4 172.16.200.1:0->172.16.200.199:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=13 ilast=7 olast=0 ad=/0
    stat: rxp=0 txp=7 rxb=0 txb=588
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=6
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-3.1 proto=0 sa=1 ref=2 serial=8 auto-negotiate
      src: 0:10.1.100.0-10.1.100.255:0
      dst: 0:192.168.4.0-192.168.4.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42923/0B replaywin=2048
           seqno=8 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42931/43200
      dec: spi=c34bb752 esp=aes key=16 3c5ceeff3cac1eaa2702b5ccb713ab9b
           ah=sha1 key=20 5903e358b3d8938ee64f0412887a0fe741ccb105
      enc: spi=b5bd4fe1 esp=aes key=16 8ae97a8abe24dae725d614d2a6efdcb0
           ah=sha1 key=20 9ec200d9c0cef9e1b7cf76e05dbf344c70f53214
      dec:pkts/bytes=0/0, enc:pkts/bytes=7/1064
    proxyid=_OCVPN2-3.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-4.1 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=11 ilast=19 olast=19 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-4.1 proto=0 sa=1 ref=2 serial=7 auto-negotiate
      src: 0:10.1.100.0-10.1.100.255:0
      dst: 0:172.16.101.0-172.16.101.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42911/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42931/43200
      dec: spi=c34bb750 esp=aes key=16 8c9844a8bcd3fda6c7bd8a4f2ec81ef1
           ah=sha1 key=20 680c7144346f5b52126cbad9f325821b048c7192
      enc: spi=f2d1f2d4 esp=aes key=16 f9625fc8590152829eb39eecab3a3999
           ah=sha1 key=20 5df8447416da541fa54dde9fa3e5c35fbfc4723f
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-4.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-3.2 ver=2 serial=3 172.16.200.1:0->172.16.200.199:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=11 ilast=6 olast=6 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-3.2 proto=0 sa=1 ref=2 serial=8 auto-negotiate
      src: 0:10.2.100.0-10.2.100.255:0
      dst: 0:192.168.5.0-192.168.5.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42923/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42930/43200
      dec: spi=c34bb753 esp=aes key=16 58ddfad9a3699f1c49f3a9f369145c28
           ah=sha1 key=20 e749c7e6a7aaff119707c792eb73cd975127873b
      enc: spi=b5bd4fe2 esp=aes key=16 8f2366e653f5f9ad6587be1ce1905764
           ah=sha1 key=20 5347bf24e51219d483c0f7b058eceab202026204
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-3.2 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-4.2 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=11 ilast=17 olast=17 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-4.2 proto=0 sa=1 ref=2 serial=7 auto-negotiate
      src: 0:10.2.100.0-10.2.100.255:0
      dst: 0:172.16.102.0-172.16.102.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42905/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42927/43200
      dec: spi=c34bb751 esp=aes key=16 41449ee5ea43d3e1f80df05fc632cd44
           ah=sha1 key=20 3ca2aea1c8764f35ccf987cdeca7cf6eb54331fb
      enc: spi=f2d1f2d5 esp=aes key=16 9010dd57e502c6296b27a4649a45a6ba
           ah=sha1 key=20 caf86a176ce04464221543f15fc3c63fc573b8ee
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-4.2 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
  • Branch_1 # get router info routing-table all
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-3.2
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-4.1
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-4.2
    S       192.168.4.0/24 [20/0] is directly connected, _OCVPN2-3.1

Troubleshooting hub-spoke with ADVPN shortcut

  • Primary-Hub # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Primary-Hub
    Server Status        : Up
    Registration time    : Sat Mar  2 11:31:54 2019
    Poll time            : Sat Mar  2 11:46:02 2019
  • Spoke1 # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Spoke
    Server Status        : Up
    Registration time    : Sat Mar  2 11:41:22 2019
    Poll time            : Sat Mar  2 11:46:44 2019
  • Primary-Hub # diagnose vpn ocvpn show-members
    Member: { "sn": "FG900D3915800083", "ip_v4": "172.16.200.4", "port": 500, "slot": 0, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Primary-Hub", "topology_role": "primary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FG100D3G15828488", "ip_v4": "172.16.200.2", "port": 500, "slot": 1, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Secondary-Hub", "topology_role": "secondary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FG100D3G15801621", "ip_v4": "172.16.200.1", "port": 500, "slot": 1000, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "10.1.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "10.2.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke1", "topology_role": "spoke" }
    Member: { "sn": "FGT51E3U16001314", "ip_v4": "172.16.200.3", "port": 500, "slot": 1001, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "192.168.4.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "192.168.5.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke2", "topology_role": "spoke" }
  • Primary-Hub # diagnose vpn ocvpn show-meta
    Topology :: auto
    License  :: full
    Members  :: 4
    Max-free :: 3
  • Primary-Hub # diagnose vpn ocvpn show-overlays
    QA
    PM
  • Spoke1 # diagnose vpn tunnel list
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=0 olast=0 ad=r/2
    stat: rxp=1 txp=34 rxb=152 txb=2856
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=46
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42895/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c7 esp=aes key=16 240e064c0f1c980ca31980b9e7605c9d
           ah=sha1 key=20 6ff022cbebcaff4c5de62eefb2e6180c40a3adb2
      enc: spi=dfcffa86 esp=aes key=16 862208de164a02af377756c2bcabd588
           ah=sha1 key=20 af6e54781fd42d7a2ba2119ec95d0f95629c8448
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=8 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=934 olast=934 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=12 olast=12 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=46
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42895/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c8 esp=aes key=16 701ec608767f4988b76c2f662464e654
           ah=sha1 key=20 93c65d106dc610d7ee3f04487f08601a9e00ffdd
      enc: spi=dfcffa87 esp=aes key=16 02b2d04dce3d81ebab69e128d45cb7ca
           ah=sha1 key=20 4a9283847f852c83a75691fad44d07d8409a2267
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=7 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=934 olast=934 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
  • Spoke1 # get router info routing-table all
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-0.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.4.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-0.1
  • Generate traffic from spoke1 to spoke2 to trigger the ADVPN shortcut and check the VPN tunnel and routing-table again on spoke1.
    branch1 # diagnose vpn tunnel list 
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0_0 ver=2 serial=a 172.16.200.1:0->172.16.200.3:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/720 options[02d0]=create_dev no-sysctl rgwy-chg frag-rfc  accept_traffic=1
    
     parent=_OCVPN2-0.0 index=0
    proxyid_num=1 child_num=0 refcnt=14 ilast=0 olast=0 ad=r/2
    stat: rxp=7 txp=7 rxb=1064 txb=588
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate add-route adr
      src: 0:10.1.100.0-10.1.100.255:0
      dst: 0:192.168.4.0-192.168.4.255:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=43180/0B replaywin=2048
           seqno=8 esn=0 replaywin_lastseq=00000008 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=43187/43200
      dec: spi=048477c9 esp=aes key=16 27c35d53793013ef24cf887561e9f313
           ah=sha1 key=20 2c8cfd328c3b29104db0ca74a00c6063f46cafe4
      enc: spi=fb9e13fd esp=aes key=16 9d0d3bf6c84b7ddaf9d9196fe74002ed
           ah=sha1 key=20 d1f541db787dea384c6a4df16fc228abeb7ae334
      dec:pkts/bytes=7/588, enc:pkts/bytes=7/1064
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=1 refcnt=12 ilast=7 olast=7 ad=r/2
    stat: rxp=2 txp=35 rxb=304 txb=2940
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=65
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42500/0B replaywin=2048
           seqno=2 esn=0 replaywin_lastseq=00000002 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c7 esp=aes key=16 240e064c0f1c980ca31980b9e7605c9d
           ah=sha1 key=20 6ff022cbebcaff4c5de62eefb2e6180c40a3adb2
      enc: spi=dfcffa86 esp=aes key=16 862208de164a02af377756c2bcabd588
           ah=sha1 key=20 af6e54781fd42d7a2ba2119ec95d0f95629c8448
      dec:pkts/bytes=1/84, enc:pkts/bytes=1/152
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=8 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=1328 olast=1328 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=5 olast=5 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=66
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42500/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42901/43200
      dec: spi=048477c8 esp=aes key=16 701ec608767f4988b76c2f662464e654
           ah=sha1 key=20 93c65d106dc610d7ee3f04487f08601a9e00ffdd
      enc: spi=dfcffa87 esp=aes key=16 02b2d04dce3d81ebab69e128d45cb7ca
           ah=sha1 key=20 4a9283847f852c83a75691fad44d07d8409a2267
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=7 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=1328 olast=1328 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=1
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    
    
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-0.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.4.0/24 [15/0] via 172.16.200.3, _OCVPN2-0.0_0
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-0.1
  • Simulate the primary hub being unavailable where all spokes' dialup VPN tunnels will switch to the secondary hub, to check VPN tunnel status and routing-table.
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=6 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=25 olast=25 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=82
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=8 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=14 olast=14 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=9
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42723/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42898/43200
      dec: spi=048477cd esp=aes key=16 9bb363a32378b5897cd42890c92df811
           ah=sha1 key=20 2ed40583b9544e37867349b4adc7c013024d7e17
      enc: spi=f345fb42 esp=aes key=16 3ea31dff3310b245700a131db4565851
           ah=sha1 key=20 522862dfb232514b845e436133b148da0e67b7c4
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=5 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=0
    
    proxyid_num=1 child_num=0 refcnt=10 ilast=19 olast=19 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=83
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=7 172.16.200.1:0->172.16.200.2:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=1 child_num=0 refcnt=11 ilast=12 olast=12 ad=r/2
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=9
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate adr
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=1a227 type=00 soft=0 mtu=1438 expire=42728/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42902/43200
      dec: spi=048477cf esp=aes key=16 b6f0ca7564abcd8559b5b0ebb3fd04c1
           ah=sha1 key=20 4130d040554b39daca72adac7583b9cc83cce3c8
      enc: spi=f345fb43 esp=aes key=16 727582f20fcedff884ba693ed2164bcd
           ah=sha1 key=20 b0a625803fde701ed9d28d256079e908954b7fc8
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    
    
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.102.0/24 [21/0] is directly connected, _OCVPN2-1.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.101.0/24 [21/0] is directly connected, _OCVPN2-1.0
    S       192.168.4.0/24 [21/0] is directly connected, _OCVPN2-1.0
    S       192.168.5.0/24 [21/0] is directly connected, _OCVPN2-1.1

Troubleshooting hub-spoke with inter-overlay source NAT

  • Primary-Hub # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Primary-Hub
    Server Status        : Up
    Registration time    : Sat Mar  2 11:31:54 2019
    Update time          : Sat Mar  2 13:57:05 2019
    Poll time            : Sat Mar  2 14:03:31 2019
  • Spoke1 # diagnose vpn ocvpn status
    Current State        : Registered
    Topology             : Dual-Hub-Spoke
    Role                 : Spoke
    Server Status        : Up
    Registration time    : Sat Mar  2 13:58:01 2019
    Poll time            : Sat Mar  2 14:04:22 2019
  • Primary-Hub # diagnose vpn ocvpn show-members
    Member: { "sn": "FG900D3915800083", "ip_v4": "172.16.200.4", "port": 500, "slot": 0, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "172.16.101.100-172.16.101.200" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "172.16.102.100-172.16.102.200" } ], "name": "Primary-Hub", "topology_role": "primary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FG100D3G15828488", "ip_v4": "172.16.200.2", "port": 500, "slot": 1, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "172.16.101.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "172.16.102.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Secondary-Hub", "topology_role": "secondary_hub", "eap": "disable", "auto_discovery": "enable" }
    Member: { "sn": "FGT51E3U16001314", "ip_v4": "172.16.200.3", "port": 500, "slot": 1001, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "192.168.4.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "192.168.5.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke2", "topology_role": "spoke" }
    Member: { "sn": "FG100D3G15801621", "ip_v4": "172.16.200.1", "port": 500, "slot": 1000, "overlay": [ { "id": 0, "name": "QA", "subnets": [ "10.1.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" }, { "id": 1, "name": "PM", "subnets": [ "10.2.100.0\/255.255.255.0" ], "ip_range": "0.0.0.0-0.0.0.0" } ], "name": "Spoke1", "topology_role": "spoke" }
  • Primary-Hub # diagnose vpn ocvpn show-meta
    Topology :: auto
    License  :: full
    Members  :: 4
    Max-free :: 3
  • Primary-Hub # diagnose vpn ocvpn show-overlays
    QA
    PM
  • Spoke1 # diagnose vpn tunnel list
    list all ipsec tunnel in vd 0
    ------------------------------------------------------
    name=_OCVPN2-0.0 ver=2 serial=c 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=3 child_num=0 refcnt=13 ilast=17 olast=17 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=29
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.0 proto=0 sa=1 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=18227 type=00 soft=0 mtu=1438 expire=42299/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42899/43200
      dec: spi=0484795d esp=aes key=16 10eeb76fadd49f00c333350d83509095
           ah=sha1 key=20 971bde5dcfca7e52fd1573cb3489e9c855f6154e
      enc: spi=dfcffaaa esp=aes key=16 d07a4dd683ee093af2dca9485aa436eb
           ah=sha1 key=20 65369be35d5ecad8cae63557318419cd6005c230
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.0_nat proto=0 sa=1 ref=2 serial=3 auto-negotiate
      src: 0:172.16.101.101-172.16.101.101:0
      dst: 0:0.0.0.0-255.255.255.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42303/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42898/43200
      dec: spi=04847961 esp=aes key=16 ea181036b02e8bc8711fb520b3e98a60
           ah=sha1 key=20 b3c449d96d5d3f090975087a62447f6918ce7930
      enc: spi=dfcffaac esp=aes key=16 f7ea5e42e9443698e6b8b32161ace40e
           ah=sha1 key=20 a7e36dd1ec0bdb6eff0aa66e442707427400c700
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.0_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.0 ver=2 serial=e 172.16.200.1:0->172.16.200.2:0 dst_mtu=0
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=10 ilast=599 olast=599 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.0 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.1.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    proxyid=_OCVPN2-1.0_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-0.1 ver=2 serial=b 172.16.200.1:0->172.16.200.4:0 dst_mtu=1500
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=3 child_num=0 refcnt=13 ilast=17 olast=17 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=1 idle=20000ms retry=3 count=0 seqno=29
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-0.1 proto=0 sa=1 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
      SA:  ref=3 options=18227 type=00 soft=0 mtu=1438 expire=42297/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42897/43200
      dec: spi=0484795e esp=aes key=16 106eaa95a2be64b566e7d1ca0aa88f6a
           ah=sha1 key=20 5dddfba7070b03d5a31931d41db06ff96e7bc542
      enc: spi=dfcffaab esp=aes key=16 29c774dbd7e54464ee298c381e71a94e
           ah=sha1 key=20 c3da7372789c0a53b3752e69baaba1a42d798820
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.1_nat proto=0 sa=1 ref=2 serial=3 auto-negotiate
      src: 0:172.16.102.101-172.16.102.101:0
      dst: 0:0.0.0.0-255.255.255.255:0
      SA:  ref=3 options=18627 type=00 soft=0 mtu=1438 expire=42307/0B replaywin=2048
           seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0
      life: type=01 bytes=0/0 timeout=42902/43200
      dec: spi=04847962 esp=aes key=16 b7daa5807cfa86906592a012a9d2478f
           ah=sha1 key=20 39c8bb4c9e3f1e9e451f22c58a172ff01155055d
      enc: spi=dfcffaad esp=aes key=16 2ecc644def4cebe6b0c4b7729da43d8e
           ah=sha1 key=20 469c6f319e83bd73468f55d430566afcd6215138
      dec:pkts/bytes=0/0, enc:pkts/bytes=0/0
    proxyid=_OCVPN2-0.1_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    ------------------------------------------------------
    name=_OCVPN2-1.1 ver=2 serial=d 172.16.200.1:0->172.16.200.2:0 dst_mtu=0
    bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/528 options[0210]=create_dev frag-rfc  accept_traffic=1
    
    proxyid_num=2 child_num=0 refcnt=10 ilast=599 olast=599 ad=/0
    stat: rxp=0 txp=0 rxb=0 txb=0
    dpd: mode=on-idle on=0 idle=20000ms retry=3 count=0 seqno=0
    natt: mode=none draft=0 interval=0 remote_port=0
    proxyid=_OCVPN2-1.1 proto=0 sa=0 ref=2 serial=1 auto-negotiate
      src: 0:10.2.100.0/255.255.255.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
    proxyid=_OCVPN2-1.1_nat proto=0 sa=0 ref=2 serial=2 auto-negotiate
      src: 0:0.0.0.0/0.0.0.0:0
      dst: 0:0.0.0.0/0.0.0.0:0
  • Spoke1 # get router info routing-table all
    Routing table for VRF=0
    Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
           O - OSPF, IA - OSPF inter area
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
           * - candidate default
    
    S*      0.0.0.0/0 [10/0] via 172.16.200.254, port1
    C       10.1.100.0/24 is directly connected, dmz
    C       10.2.100.0/24 is directly connected, loop
    C       11.101.1.0/24 is directly connected, wan1
    C       11.102.1.0/24 is directly connected, wan2
    S       172.16.101.0/24 [20/0] is directly connected, _OCVPN2-0.1
    C       172.16.101.101/32 is directly connected, _OCVPN2-0.1
    C       172.16.200.0/24 is directly connected, port1
    S       172.16.102.0/24 [20/0] is directly connected, _OCVPN2-0.0
    C       172.16.102.101/32 is directly connected, _OCVPN2-0.0
    S       192.168.4.0/24 [20/0] is directly connected, _OCVPN2-0.0
    S       192.168.5.0/24 [20/0] is directly connected, _OCVPN2-0.1
  • Spoke1 # show firewall policy
     ..............................
    
        edit 9
            set name "_OCVPN2-1.1_nat"
            set uuid 3f7a84b8-3d36-51e9-ee97-8f418c91e666
            set srcintf "any"
            set dstintf "_OCVPN2-1.1"
            set srcaddr "all"
            set dstaddr "_OCVPN2-1.1_remote_networks"
            set action accept
            set schedule "always"
            set service "ALL"
            set comments "Generated by OCVPN Cloud Service."
            set nat enable
        next
        edit 12
            set name "_OCVPN2-1.0_nat"
            set uuid 3fafec98-3d36-51e9-80c0-5d99325bad83
            set srcintf "any"
            set dstintf "_OCVPN2-1.0"
            set srcaddr "all"
            set dstaddr "_OCVPN2-1.0_remote_networks"
            set action accept
            set schedule "always"
            set service "ALL"
            set comments "Generated by OCVPN Cloud Service."
            set nat enable
        next
       .................................