VXLAN over IPsec using a VXLAN tunnel endpoint

This example describes how to implement VXLAN over IPsec VPN using a VXLAN tunnel endpoint (VTEP).

This example uses a hub and spoke topology. Dialup VPN is used because it allows a single phase 1 dialup definition on the hub FortiGate. Additional spoke tunnels are added with minimal changes to the hub by adding a user account and VXLAN interface for each spoke. Spoke-to-spoke communication is established through the hub. This example assumes that the authentication users and user groups have already been created. While this topology demonstrates hub and spoke with dialup tunnels with XAuth authentication, the same logic can be applied to a static VPN with or without XAuth.

IPsec tunnel interfaces are used to support VXLAN tunnel termination. An IP address is set for each tunnel interface. Ping access is allowed for troubleshooting purposes.

VTEPs are created on the hub and each spoke to forward VXLAN traffic through the IPsec tunnels. VXLAN encapsulates OSI layer 2 Ethernet frames within layer 3 IP packets. You will need to either combine the internal port1 and VXLAN interface into a soft switch, or create a virtual wire pair so that devices behind port1 have direct layer 2 access to remote peers over the VXLAN tunnel. This example uses a switch interface on the hub and a virtual wire pair on the spokes to demonstrate the two different methods.

In order to apply an IPsec VPN interface on the VXLAN interface setting, net-device must be disabled in the IPsec VPN phase 1 settings.

To configure the hub FortiGate:

1. Configure the IPsec phase 1 interface:

   config vpn ipsec phase1-interface
       edit "SPOKES"
           set type dynamic
           set interface "port2"
           set mode aggressive
           set peertype one
           set net-device disable
           set proposal aes256-sha256
           set xauthtype auto
           set authusrgrp "SPOKES"
           set peerid "SPOKES"
           set psksecret <secret>
       next
   end
   

2. Configure the IPsec phase 2 interface:

   config vpn ipsec phase2-interface
       edit "SPOKES"
           set phase1name "SPOKES"
           set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
       next
   end

3. Configure the IPsec VPN policy that allows VXLAN traffic between the spokes:

   config firewall policy
       edit 1
           set name "VXLAN_SPOKE_to_SPOKE"
           set srcintf "SPOKES"
           set dstintf "SPOKES"
           set srcaddr "NET_192.168.255.0"
           set dstaddr "NET_192.168.255.0"
           set action accept
           set schedule "always"
           set service "UDP_4789"
           set logtraffic all
           set fsso disable
       next
   end

4. Configure the IPsec tunnel interfaces (the remote IP address is not used, but it is necessary for this configuration):

   config system interface
       edit "SPOKES"
           set vdom "root"
           set ip 192.168.255.1 255.255.255.255
           set allowaccess ping
           set type tunnel
           set remote-ip 192.168.255.254 255.255.255.0
           set snmp-index 12
           set interface "port2"
       next
   end

5. Configure the VXLAN interfaces. Each spoke requires a VXLAN interface with a different VNI. The remote IP is the tunnel interfaces IP of the spokes.

   1. Spoke 1:

      config system VXLAN
          edit "SPOKES_VXLAN1"
              set interface "SPOKES"
              set vni 1
              set remote-ip "192.168.255.2"
          next
      end

   2. Spoke 2:

      config system VXLAN
          edit "SPOKES_VXLAN2"
              set interface "SPOKES"
              set vni 2
              set remote-ip "192.168.255.3"
          next
      end

To configure the spoke FortiGates:

1. Configure the IPsec phase 1 interface:

   config vpn ipsec phase1-interface
       edit "HUB"
           set interface "port2"
           set mode aggressive
           set peertype any
           set net-device disable
           set proposal aes256-sha256
           set localid "SPOKES"
           set xauthtype client
           set authusr "SPOKE1"
           set authpasswd <secret>
           set remote-gw <hub public IP>
           set psksecret <secret>
       next
   end

2. Configure the IPsec phase 2 interface:

   config vpn ipsec phase2-interface
       edit "HUB"
           set phase1name "HUB"
           set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
           set auto-negotiate enable
           set src-subnet 192.168.255.2 255.255.255.255
       next
   end

   The hub FortiGate inserts a reverse route pointing to newly established tunnel interfaces for any of the subnets that the spoke FortiGate's source quick mode selectors provides. This is why you should set the tunnel IP address here.

3. Configure the IPsec VPN policy:

   config firewall policy
       edit 1
           set name "VTEP_IPSEC_POLICY"
           set srcintf "HUB"
           set dstintf "HUB"
           set srcaddr "none"
           set dstaddr "none"
           set action accept
           set schedule "always"
           set service "PING"
           set logtraffic disable
           set fsso disable
       next
   end

4. Configure the IPsec tunnel interface:

   config system interface
       edit "HUB"
           set vdom "root"
           set ip 192.168.255.2 255.255.255.255
           set allowaccess ping
           set type tunnel
           set remote-ip 192.168.255.1 255.255.255.0
           set snmp-index 12
           set interface "port2"
       next
   end

5. Configure the VXLAN interfaces (the remote IP is the tunnel interface IP of the hub):

   1. Spoke 1:

      config system VXLAN
          edit "HUB_VXLAN"
              set interface "HUB"
              set vni 1
              set remote-ip "192.168.255.1"
          next
      end

   2. Spoke 2:

      config system VXLAN
          edit "HUB_VXLAN"
              set interface "HUB"
              set vni 2
              set remote-ip "192.168.255.1"
          next
      end
      

To bind the VXLAN interface to the internal interface:

1. Configure a switch interface on the hub:

   config system switch-interface
       edit "SW"
           set vdom "root"
           set member "port1" "SPOKES_VXLAN1" "SPOKES_VXLAN2"
           set intra-switch-policy {implicit | explicit}
       next
   end

   Allowing intra-switch traffic is implicitly allowed by default. Use set intra-switch-policy explicit to require firewall policies to allow traffic between switch interfaces.

2. Configure a virtual wire pair on the spokes:

   config system virtual-wire-pair
       edit "VWP"
           set member "HUB_VXLAN" "port1"
       next
   end

   The virtual wire pair requires an explicit policy to allow traffic between interfaces.

To test the configuration:

1. Ping the hub FortiGate from the spoke FortiGate:

   user@pc-spoke1:~$ ping 192.168.1.1 -c 3
   PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
   64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.24 ms
   64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.672 ms
   64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.855 ms
   --- 192.168.1.1 ping statistics ---
   3 packets transmitted, 3 received, 0% packet loss, time 2002 ms
   rtt min/avg/max/mdev = 0.672/0.923/1.243/0.239 ms

2. Sniff traffic on the hub FortiGate:

   # diagnose sniffer packet any 'icmp or (udp and port 4789)' 4 0 
   interfaces=[any] filters=[icmp or (udp and port 4789)]
   15:00:01.438230 SPOKES in 192.168.255.2.4790 -> 192.168.255.1.4789: udp 106
   15:00:01.438256 SPOKES_VXLAN1 in 192.168.1.2 -> 192.168.1.1: icmp: echo request
   15:00:01.438260 port1 out 192.168.1.2 -> 192.168.1.1: icmp: echo request
   15:00:01.438532 port1 in 192.168.1.1 -> 192.168.1.2: icmp: echo reply
   15:00:01.438536 SPOKES_VXLAN1 out 192.168.1.1 -> 192.168.1.2: icmp: echo reply
   15:00:01.438546 SPOKES out 192.168.255.1.4851 -> 192.168.255.2.4789: udp 106