Fortinet black logo

Handbook

Example 1: Remote sites with different subnets

6.0.0
Copy Link
Copy Doc ID 4afb0436-a998-11e9-81a4-00505692583a:496458
Download PDF

Example 1: Remote sites with different subnets

This example provides a configuration example for IPsec VPN tunnels between three FortiGate in transparent Mode in different subnets, as well as some troubleshooting steps.

The expectation for this example is that PC1 will be able to communicate via the IPsec tunnels with PC2 and PC3, which are in different subnets.

The requirements for this example are:

  • Because both FortiGate are not in the same broadcast domain (separated by routers), the hosts on each side must be on different subnets.
  • FortiGate management IP addresses must be in the same subnet as the local hosts
  • The default gateways (router1 ,router2, router3) for PC1 , PC2 and PC3 must be behind port2 in order for the FortiGate to match the appropriate Encrypt firewall policy (port1 --> port2)

Configuration of FortiGate 1 (FGT1):

Only relevant parts of configuration are provided.

config system settings

set opmode transparent

set manageip 10.1.1.100/255.255.255.0

end

config router static

edit 1

set gateway 10.1.1.254

next

end

config firewall address

edit "10.1.1.0/24"

set subnet 10.1.1.0 255.255.255.0

next

edit "10.2.2.0/24"

set subnet 10.2.2.0 255.255.255.0

next

edit "10.3.3.0/24"

set subnet 10.3.3.0 255.255.255.0

next

end

config vpn ipsec phase1

edit "to_FGT2"

set proposal 3des-sha1 aes128-sha1 des-md5

set remote-gw 10.2.2.100

set psksecret fortinet

next

edit "to_FGT3"

set proposal 3des-sha1 aes128-sha1 des-md5

set remote-gw 10.3.3.100

set psksecret fortinet

next

end

config vpn ipsec phase2

edit "to_FGT2"

set keepalive enable

set phase1name "to_FGT2"

set proposal 3des-sha1 aes128-sha1

set dst-subnet 10.2.2.0 255.255.255.0

set src-subnet 10.1.1.0 255.255.255.0

next

edit "to_FGT3"

set keepalive enable

set phase1name "to_FGT3"

set proposal 3des-sha1 aes128-sha1

set dst-subnet 10.3.3.0 255.255.255.0

set src-subnet 10.1.1.0 255.255.255.0

next

end

config firewall policy

edit 1

set srcintf "port1"

set dstintf "port2"

set srcaddr "10.1.1.0/24"

set dstaddr "10.2.2.0/24"

set action ipsec

set schedule "always"

set service "ALL"

set inbound enable

set outbound enable

set vpntunnel "to_FGT2"

next

edit 3

set srcintf "port1"

set dstintf "port2"

set srcaddr "10.1.1.0/24"

set dstaddr "10.3.3.0/24"

set action ipsec

set schedule "always"

set service "ALL"

set inbound enable

set outbound enable

set vpntunnel "to_FGT3"

next

end

Configuration of FortiGate 2 (FGT2):

Only relevant parts of configuration are provided.

config system settings

set opmode transparent

set manageip 10.2.2.100/255.255.255.0

end

config router static

edit 1

set gateway 10.2.2.254

next

end

config firewall address

edit "10.1.1.0/24"

set subnet 10.1.1.0 255.255.255.0

next

edit "10.2.2.0/24"

set subnet 10.2.2.0 255.255.255.0

next

end

config vpn ipsec phase1

edit "to_FGT1"

set nattraversal disable

set proposal 3des-sha1 aes128-sha1 des-md5

set remote-gw 10.1.1.100

set psksecret fortinet

next

end

config vpn ipsec phase2

edit "to_FGT1"

set keepalive enable

set phase1name "to_FGT1"

set proposal 3des-sha1 aes128-sha1

set dst-subnet 10.1.1.0 255.255.255.0

set src-subnet 10.2.2.0 255.255.255.0

next

end

config firewall policy

edit 1

set srcintf "port1"

set dstintf "port2"

set srcaddr "10.2.2.0/24"

set dstaddr "10.1.1.0/24"

set action ipsec

set schedule "always"

set service "ALL"

set inbound enable

set outbound enable

set vpntunnel "to_FGT1"

next

end

Troubleshooting procedure

All steps given when PC1 pings PC2.

Verify if IPsec tunnels are up

FGT1 # diagnose vpn tunnel list

list all ipsec tunnel in vd 0

------------------------------------------------------

name=to_FGT2 ver=0 serial=1 10.1.1.100:0->10.2.2.100:0 lgwy=dyn tun=tunnel mode= auto

bound_if=0

proxyid_num=1 child_num=0 refcnt=7 ilast=0 olast=0

stat: rxp=0 txp=0 rxb=0 txb=0

dpd: mode=active on=1 idle=5000ms retry=3 count=0 seqno=1455

natt: mode=none draft=0 interval=0 remote_port=0

proxyid=to_FGT2 proto=0 sa=0 ref=1 auto_negotiate=0 serial=5

src: 10.1.1.0/255.255.255.0:0

dst: 10.2.2.0/255.255.255.0:0

The above tunnel is down (output given as example)!

FGT2 # diagnose vpn tunnel list

list all ipsec tunnel in vd 0

------------------------------------------------------

name=to_FGT1 10.2.2.100:0->10.1.1.100:0 lgwy=dyn tun=tunnel mode=auto bound_if=0

proxyid_num=1 child_num=0 refcnt=7 ilast=1 olast=1

stat: rxp=0 txp=0 rxb=0 txb=0

dpd: mode=active on=1 idle=5000ms retry=3 count=0 seqno=21 natt:

mode=none draft=0 interval=0 remote_port=0

proxyid=to_FGT1 proto=0 sa=1 ref=2 auto_negotiate=0 serial=1

src: 10.2.2.0/255.255.255.0:0

dst: 10.1.1.0/255.255.255.0:0

SA: ref=3 options=00000009 type=00 soft=0 mtu=1436 expire=1771 replaywin=0 seq no=1

life: type=01 bytes=0/0 timeout=1773/1800

dec: spi=c1a8e951 esp=3des key=24 9213fdf22b150e01abb3535d1a647044eebf772b92f2f7ee

ah=sha1 key=20 66a38bf99f0b2d234f64b5a05187995c4f56f6bb

enc: spi=322067b4 esp=3des key=24 720e5680329937fb3630b7ed70bd41bb3114d3c269ae8b61

ah=sha1 key=20 e316113eb6ea03b014b3a5f9c1a3bd386637801a

The above tunnel is up!

Verify that destination local hosts are seen in the ARP table (necessary for IPsec despite being in TP mode)

FGT2 # get system arp

Address Age(min) Hardware Addr Interface

10.2.2.10 2 00:50:56:00:76:04 root.b

10.2.2.254 0 00:09:0f:30:29:e4 root.b

Using the debug flow command on the initiator side (example on FortiGate1)

FGT1 # diagnose debug flow filter addr 10.1.1.10

FGT1 # diagnose debug flow show console enable

FGT1 # diagnose debug enable

FGT1 # diagnose debug flow trace start 50

FGT1 # id=36870 trace_id=615 msg="vd-root received a packet(proto=1, 10.1.1.10:512-

>10.2.2.10:8) from port1."

id=36870 trace_id=615 msg="allocate a new session-00000636"

id=36870 trace_id=615 msg="Allowed by Policy-1: encrypt"

id=36870 trace_id=615 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=615 msg="SA is not ready yet, drop"

id=36870 trace_id=616 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port1."

id=36870 trace_id=616 msg="Find an existing session, id-00000636, original direction"

id=36870 trace_id=616 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=616 msg="encrypted, and send to 10.2.2.100 with source 10.1.1.100"

id=36870 trace_id=616 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e0"

id=36870 trace_id=617 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port1."

id=36870 trace_id=617 msg="Find an existing session, id-00000636, original direction"

id=36870 trace_id=617 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=617 msg="encrypted, and send to 10.2.2.100 with source 10.1.1.100"

id=36870 trace_id=617 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e0"

id=36870 trace_id=618 msg="vd-root received a packet(proto=1, 10.2.2.10:512->10.1.1.10:0) from port2."

id=36870 trace_id=618 msg="Find an existing session, id-00000636, reply direction"

id=36870 trace_id=618 msg="send out via dev-port1, dst-mac-00:50:56:00:76:03"

id=36870 trace_id=619 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port1."

id=36870 trace_id=619 msg="Find an existing session, id-00000636, original direction"

id=36870 trace_id=619 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=619 msg="encrypted, and send to 10.2.2.100 with source 10.1.1.100"

id=36870 trace_id=619 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e0"

id=36870 trace_id=620 msg="vd-root received a packet(proto=1, 10.2.2.10:512->10.1.1.10:0) from port2."

id=36870 trace_id=620 msg="Find an existing session, id-00000636, reply direction"

id=36870 trace_id=620 msg="send out via dev-port1, dst-mac-00:50:56:00:76:03"

The message "id=36870 trace_id=615 msg="SA is not ready yet, drop" simply means that the tunnel was not up yet.
Using the debug flow command on the receiver side (example on FortiGate2)

FGT2 # diagnose debug flow filter addr 10.1.1.10

FGT2 # diagnose debug flow show console enable

FGT2 # diagnose debug enable

FGT2 # diagnose debug flow trace start 50

FGT2 # id=36870 trace_id=51 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port2."

id=36870 trace_id=51 msg="allocate a new session-00000435"

id=36870 trace_id=51 msg="Allowed by Policy-1:"

id=36870 trace_id=51 msg="send out via dev-port1, dst-mac-00:50:56:00:76:04"

id=36870 trace_id=52 msg="vd-root received a packet(proto=1, 10.2.2.10:512->10.1.1.10:0) from port1."

id=36870 trace_id=52 msg="Find an existing session, id-00000435, reply direction"

id=36870 trace_id=52 msg="enter IPsec tunnel-to_FGT1"

id=36870 trace_id=52 msg="encrypted, and send to 10.1.1.100 with source 10.2.2.100"

id=36870 trace_id=52 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e4"

Using the sniffer trace (example on FortiGate2)

FGT2 # diagnose sniffer packet any "host 10.2.2.10" 4

9.460021 root.b out arp who-has 10.2.2.10 tell 10.2.2.100

9.460028 port2 out arp who-has 10.2.2.10 tell 10.2.2.100

9.460034 port1 out arp who-has 10.2.2.10 tell 10.2.2.100

9.460462 port1 in arp reply 10.2.2.10 is-at 0:50:56:0:76:4

9.460462 root.b in arp reply 10.2.2.10 is-at 0:50:56:0:76:4

[...]

49.477368 port2 in 10.1.1.10 -> 10.2.2.10: icmp: echo request

49.477444 port1 out 10.1.1.10 -> 10.2.2.10: icmp: echo request

49.477898 port1 in 10.2.2.10 -> 10.1.1.10: icmp: echo reply

50.510023 port2 in 10.1.1.10 -> 10.2.2.10: icmp: echo request

50.510079 port1 out 10.1.1.10 -> 10.2.2.10: icmp: echo request

50.510524 port1 in 10.2.2.10 -> 10.1.1.10: icmp: echo reply

The above ARP process in transparent mode with IPsec is allowing the FortiGate to:
  • Identify the MAC address of the destination device 10.2.2.10
  • Populate the MAC table (see below), which in turn will give a destination interface and allow a Firewall policy look-up
Check the FDB entries for the destination

FGT2 # diagnose netlink brctl name host root.b

port no device devname mac addr ttl

[...]

1 2 port1 00:50:56:00:76:04 0

Example 1: Remote sites with different subnets

This example provides a configuration example for IPsec VPN tunnels between three FortiGate in transparent Mode in different subnets, as well as some troubleshooting steps.

The expectation for this example is that PC1 will be able to communicate via the IPsec tunnels with PC2 and PC3, which are in different subnets.

The requirements for this example are:

  • Because both FortiGate are not in the same broadcast domain (separated by routers), the hosts on each side must be on different subnets.
  • FortiGate management IP addresses must be in the same subnet as the local hosts
  • The default gateways (router1 ,router2, router3) for PC1 , PC2 and PC3 must be behind port2 in order for the FortiGate to match the appropriate Encrypt firewall policy (port1 --> port2)

Configuration of FortiGate 1 (FGT1):

Only relevant parts of configuration are provided.

config system settings

set opmode transparent

set manageip 10.1.1.100/255.255.255.0

end

config router static

edit 1

set gateway 10.1.1.254

next

end

config firewall address

edit "10.1.1.0/24"

set subnet 10.1.1.0 255.255.255.0

next

edit "10.2.2.0/24"

set subnet 10.2.2.0 255.255.255.0

next

edit "10.3.3.0/24"

set subnet 10.3.3.0 255.255.255.0

next

end

config vpn ipsec phase1

edit "to_FGT2"

set proposal 3des-sha1 aes128-sha1 des-md5

set remote-gw 10.2.2.100

set psksecret fortinet

next

edit "to_FGT3"

set proposal 3des-sha1 aes128-sha1 des-md5

set remote-gw 10.3.3.100

set psksecret fortinet

next

end

config vpn ipsec phase2

edit "to_FGT2"

set keepalive enable

set phase1name "to_FGT2"

set proposal 3des-sha1 aes128-sha1

set dst-subnet 10.2.2.0 255.255.255.0

set src-subnet 10.1.1.0 255.255.255.0

next

edit "to_FGT3"

set keepalive enable

set phase1name "to_FGT3"

set proposal 3des-sha1 aes128-sha1

set dst-subnet 10.3.3.0 255.255.255.0

set src-subnet 10.1.1.0 255.255.255.0

next

end

config firewall policy

edit 1

set srcintf "port1"

set dstintf "port2"

set srcaddr "10.1.1.0/24"

set dstaddr "10.2.2.0/24"

set action ipsec

set schedule "always"

set service "ALL"

set inbound enable

set outbound enable

set vpntunnel "to_FGT2"

next

edit 3

set srcintf "port1"

set dstintf "port2"

set srcaddr "10.1.1.0/24"

set dstaddr "10.3.3.0/24"

set action ipsec

set schedule "always"

set service "ALL"

set inbound enable

set outbound enable

set vpntunnel "to_FGT3"

next

end

Configuration of FortiGate 2 (FGT2):

Only relevant parts of configuration are provided.

config system settings

set opmode transparent

set manageip 10.2.2.100/255.255.255.0

end

config router static

edit 1

set gateway 10.2.2.254

next

end

config firewall address

edit "10.1.1.0/24"

set subnet 10.1.1.0 255.255.255.0

next

edit "10.2.2.0/24"

set subnet 10.2.2.0 255.255.255.0

next

end

config vpn ipsec phase1

edit "to_FGT1"

set nattraversal disable

set proposal 3des-sha1 aes128-sha1 des-md5

set remote-gw 10.1.1.100

set psksecret fortinet

next

end

config vpn ipsec phase2

edit "to_FGT1"

set keepalive enable

set phase1name "to_FGT1"

set proposal 3des-sha1 aes128-sha1

set dst-subnet 10.1.1.0 255.255.255.0

set src-subnet 10.2.2.0 255.255.255.0

next

end

config firewall policy

edit 1

set srcintf "port1"

set dstintf "port2"

set srcaddr "10.2.2.0/24"

set dstaddr "10.1.1.0/24"

set action ipsec

set schedule "always"

set service "ALL"

set inbound enable

set outbound enable

set vpntunnel "to_FGT1"

next

end

Troubleshooting procedure

All steps given when PC1 pings PC2.

Verify if IPsec tunnels are up

FGT1 # diagnose vpn tunnel list

list all ipsec tunnel in vd 0

------------------------------------------------------

name=to_FGT2 ver=0 serial=1 10.1.1.100:0->10.2.2.100:0 lgwy=dyn tun=tunnel mode= auto

bound_if=0

proxyid_num=1 child_num=0 refcnt=7 ilast=0 olast=0

stat: rxp=0 txp=0 rxb=0 txb=0

dpd: mode=active on=1 idle=5000ms retry=3 count=0 seqno=1455

natt: mode=none draft=0 interval=0 remote_port=0

proxyid=to_FGT2 proto=0 sa=0 ref=1 auto_negotiate=0 serial=5

src: 10.1.1.0/255.255.255.0:0

dst: 10.2.2.0/255.255.255.0:0

The above tunnel is down (output given as example)!

FGT2 # diagnose vpn tunnel list

list all ipsec tunnel in vd 0

------------------------------------------------------

name=to_FGT1 10.2.2.100:0->10.1.1.100:0 lgwy=dyn tun=tunnel mode=auto bound_if=0

proxyid_num=1 child_num=0 refcnt=7 ilast=1 olast=1

stat: rxp=0 txp=0 rxb=0 txb=0

dpd: mode=active on=1 idle=5000ms retry=3 count=0 seqno=21 natt:

mode=none draft=0 interval=0 remote_port=0

proxyid=to_FGT1 proto=0 sa=1 ref=2 auto_negotiate=0 serial=1

src: 10.2.2.0/255.255.255.0:0

dst: 10.1.1.0/255.255.255.0:0

SA: ref=3 options=00000009 type=00 soft=0 mtu=1436 expire=1771 replaywin=0 seq no=1

life: type=01 bytes=0/0 timeout=1773/1800

dec: spi=c1a8e951 esp=3des key=24 9213fdf22b150e01abb3535d1a647044eebf772b92f2f7ee

ah=sha1 key=20 66a38bf99f0b2d234f64b5a05187995c4f56f6bb

enc: spi=322067b4 esp=3des key=24 720e5680329937fb3630b7ed70bd41bb3114d3c269ae8b61

ah=sha1 key=20 e316113eb6ea03b014b3a5f9c1a3bd386637801a

The above tunnel is up!

Verify that destination local hosts are seen in the ARP table (necessary for IPsec despite being in TP mode)

FGT2 # get system arp

Address Age(min) Hardware Addr Interface

10.2.2.10 2 00:50:56:00:76:04 root.b

10.2.2.254 0 00:09:0f:30:29:e4 root.b

Using the debug flow command on the initiator side (example on FortiGate1)

FGT1 # diagnose debug flow filter addr 10.1.1.10

FGT1 # diagnose debug flow show console enable

FGT1 # diagnose debug enable

FGT1 # diagnose debug flow trace start 50

FGT1 # id=36870 trace_id=615 msg="vd-root received a packet(proto=1, 10.1.1.10:512-

>10.2.2.10:8) from port1."

id=36870 trace_id=615 msg="allocate a new session-00000636"

id=36870 trace_id=615 msg="Allowed by Policy-1: encrypt"

id=36870 trace_id=615 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=615 msg="SA is not ready yet, drop"

id=36870 trace_id=616 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port1."

id=36870 trace_id=616 msg="Find an existing session, id-00000636, original direction"

id=36870 trace_id=616 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=616 msg="encrypted, and send to 10.2.2.100 with source 10.1.1.100"

id=36870 trace_id=616 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e0"

id=36870 trace_id=617 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port1."

id=36870 trace_id=617 msg="Find an existing session, id-00000636, original direction"

id=36870 trace_id=617 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=617 msg="encrypted, and send to 10.2.2.100 with source 10.1.1.100"

id=36870 trace_id=617 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e0"

id=36870 trace_id=618 msg="vd-root received a packet(proto=1, 10.2.2.10:512->10.1.1.10:0) from port2."

id=36870 trace_id=618 msg="Find an existing session, id-00000636, reply direction"

id=36870 trace_id=618 msg="send out via dev-port1, dst-mac-00:50:56:00:76:03"

id=36870 trace_id=619 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port1."

id=36870 trace_id=619 msg="Find an existing session, id-00000636, original direction"

id=36870 trace_id=619 msg="enter IPsec tunnel-to_FGT2"

id=36870 trace_id=619 msg="encrypted, and send to 10.2.2.100 with source 10.1.1.100"

id=36870 trace_id=619 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e0"

id=36870 trace_id=620 msg="vd-root received a packet(proto=1, 10.2.2.10:512->10.1.1.10:0) from port2."

id=36870 trace_id=620 msg="Find an existing session, id-00000636, reply direction"

id=36870 trace_id=620 msg="send out via dev-port1, dst-mac-00:50:56:00:76:03"

The message "id=36870 trace_id=615 msg="SA is not ready yet, drop" simply means that the tunnel was not up yet.
Using the debug flow command on the receiver side (example on FortiGate2)

FGT2 # diagnose debug flow filter addr 10.1.1.10

FGT2 # diagnose debug flow show console enable

FGT2 # diagnose debug enable

FGT2 # diagnose debug flow trace start 50

FGT2 # id=36870 trace_id=51 msg="vd-root received a packet(proto=1, 10.1.1.10:512->10.2.2.10:8) from port2."

id=36870 trace_id=51 msg="allocate a new session-00000435"

id=36870 trace_id=51 msg="Allowed by Policy-1:"

id=36870 trace_id=51 msg="send out via dev-port1, dst-mac-00:50:56:00:76:04"

id=36870 trace_id=52 msg="vd-root received a packet(proto=1, 10.2.2.10:512->10.1.1.10:0) from port1."

id=36870 trace_id=52 msg="Find an existing session, id-00000435, reply direction"

id=36870 trace_id=52 msg="enter IPsec tunnel-to_FGT1"

id=36870 trace_id=52 msg="encrypted, and send to 10.1.1.100 with source 10.2.2.100"

id=36870 trace_id=52 msg="send out via dev-port2, dst-mac-00:09:0f:30:29:e4"

Using the sniffer trace (example on FortiGate2)

FGT2 # diagnose sniffer packet any "host 10.2.2.10" 4

9.460021 root.b out arp who-has 10.2.2.10 tell 10.2.2.100

9.460028 port2 out arp who-has 10.2.2.10 tell 10.2.2.100

9.460034 port1 out arp who-has 10.2.2.10 tell 10.2.2.100

9.460462 port1 in arp reply 10.2.2.10 is-at 0:50:56:0:76:4

9.460462 root.b in arp reply 10.2.2.10 is-at 0:50:56:0:76:4

[...]

49.477368 port2 in 10.1.1.10 -> 10.2.2.10: icmp: echo request

49.477444 port1 out 10.1.1.10 -> 10.2.2.10: icmp: echo request

49.477898 port1 in 10.2.2.10 -> 10.1.1.10: icmp: echo reply

50.510023 port2 in 10.1.1.10 -> 10.2.2.10: icmp: echo request

50.510079 port1 out 10.1.1.10 -> 10.2.2.10: icmp: echo request

50.510524 port1 in 10.2.2.10 -> 10.1.1.10: icmp: echo reply

The above ARP process in transparent mode with IPsec is allowing the FortiGate to:
  • Identify the MAC address of the destination device 10.2.2.10
  • Populate the MAC table (see below), which in turn will give a destination interface and allow a Firewall policy look-up
Check the FDB entries for the destination

FGT2 # diagnose netlink brctl name host root.b

port no device devname mac addr ttl

[...]

1 2 port1 00:50:56:00:76:04 0