Fortinet black logo

Administration Guide

VXLAN over IPsec using a VXLAN tunnel endpoint

VXLAN over IPsec using a VXLAN tunnel endpoint

This example describes how to implement VXLAN over IPsec VPN using a VXLAN tunnel endpoint (VTEP).

This example shows a specific configuration that uses a hub-and-spoke topology. However, the same logic can be applied to a static VPN with or without XAuth. In this hub-and-spoke topology, dialup VPN is convenient because it uses a single phase 1 dialup definition on the hub FortiGate. Additional spoke tunnels are added without any changes to the hub, other than adding a user account for each additional spoke. Spoke-to-spoke communication is established through the hub. This example assumes the authentication users and user groups have already been created.

IPsec tunnel interfaces are used to support VXLAN tunnel termination. An IP address is set for each tunnel interface. Ping access is allowed for troubleshooting purposes.

VTEPs are created on each of the hub and spokes in order to forward VXLAN traffic through the IPsec tunnels. VXLAN encapsulates OSI layer 2 Ethernet frames within layer 3 IP packets. You will need to either combine the internal port and VXLAN interface into a soft switch, or create a virtual wire pair so that devices behind port1 have direct layer 2 access to remote peers over the VXLAN tunnel. This example uses a switch interface on the hub and a virtual wire pair on the spokes to demonstrate the two different methods.

Finally, in order to apply an IPsec VPN interface on the VXLAN interface setting, net-device must be disabled in the IPsec VPN phase 1 settings. All VXLAN interfaces in this example share the same VXLAN network ID (vni).

To configure the hub FortiGate:
  1. Configure the phase 1 and phase 2 interfaces:
    config vpn ipsec phase1-interface
       edit "SPOKES"
          set type dynamic
          set interface "port2"
          set mode aggressive
          set peertype one
          set net-device disable
          set proposal aes256-sha256
          set xauthtype auto
          set authusrgrp "SPOKES"
          set peerid "SPOKES"
          set psksecret <secret>
       next
    end
    config vpn ipsec phase2-interface
       edit "SPOKES"
          set phase1name "SPOKES"
          set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
       next
    end
  2. Configure the IPsec VPN policy that allows VXLAN traffic between spokes:
    config firewall policy
       edit 1
          set name "VXLAN_SPOKE_to_SPOKE"
          set srcintf "SPOKES"
          set dstintf "SPOKES"
          set srcaddr "NET_192.168.255.0"
          set dstaddr "NET_192.168.255.0"
          set action accept
          set schedule "always"
          set service "UDP_4789"
          set logtraffic all
          set fsso disable
       next
    end
  3. Configure the IPsec tunnel interfaces (the remote IP address is not used, but it is necessary for this configuration):
    config system interface
       edit "SPOKES"
          set vdom "root"
          set ip 192.168.255.1 255.255.255.255
          set allowaccess ping
          set type tunnel
          set remote-ip 192.168.255.254 255.255.255.0
          set snmp-index 12
          set interface "port2"
       next
    end
  4. Configure the VXLAN interface (the remote IP is the tunnel interfaces IPs of the spokes):
    config system VXLAN
       edit "SPOKES_VXLAN"
          set interface "SPOKES"
          set vni 1
          set remote-ip "192.168.255.2" "192.168.255.3"
       next
    end
To configure the spoke FortiGates:
  1. Configure the phase 1 and phase 2 interfaces:
    config vpn ipsec phase1-interface
       edit "HUB"
          set interface "port2"
          set mode aggressive
          set peertype any
          set net-device disable
          set proposal aes256-sha256
          set localid "SPOKES"
          set xauthtype client
          set authusr "SPOKE1"
          set authpasswd <secret>
          set remote-gw <hub public IP>
          set psksecret <secret>
       next
    end
    config vpn ipsec phase2-interface
       edit "HUB"
          set phase1name "HUB"
          set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
          set auto-negotiate enable
          set src-subnet 192.168.255.2 255.255.255.255
       next
    end
    Note

    The hub FortiGate inserts a reverse route pointing to newly established tunnel interfaces for any of the subnets that the spoke FortiGate's source quick mode selectors provides. This is why you should set the tunnel IP address here.

  2. Configure the IPsec VPN policy:
    config firewall policy
       edit 1
          set name "VTEP_IPSEC_POLICY"
          set srcintf "HUB"
          set dstintf "HUB"
          set srcaddr "none"
          set dstaddr "none"
          set action accept
          set schedule "always"
          set service "PING"
          set logtraffic disable
          set fsso disable
       next
    end
  3. Configure the IPsec tunnel interfaces:
    config system interface
       edit "HUB"
          set vdom "root"
          set ip 192.168.255.2 255.255.255.255
          set allowaccess ping
          set type tunnel
          set remote-ip 192.168.255.1 255.255.255.0
          set snmp-index 12
          set interface "port2"
       next
    end
  4. Configure the VXLAN interface (the remote IP is the tunnel interface IP of the hub):
    config system VXLAN
       edit "HUB_VXLAN"
          set interface "HUB"
          set vni 1
          set remote-ip "192.168.255.1"
       next
    end

    To establish a VXLAN tunnel between spokes, you can add a spoke's tunnel IP address in remote-ip.

    Note

    To add more remote IP addresses to a VXLAN interface, the interface cannot be in use. You may want to provision future spokes' remote IP addresses at this point to avoid traffic disruption. Otherwise, you must delete the reference (the policy in this example) before adding remote IP addresses.

To bind the VXLAN interface to the internal interface:
  1. Configure a switch interface on the hub:
    config system switch-interface
        edit "SW"
            set vdom "root"
            set member "port1" "SPOKES_VXLAN"
            set intra-switch-policy {implicit | explicit}
        next
    end

    Note

    Allowing intra-switch traffic is implicitly allowed by default. Use set intra-switch-policy explicit to require firewall policies to allow traffic between switch interfaces.

  2. Configure a virtual wire pair on the spokes:
    config system virtual-wire-pair
        edit "VWP"
            set member "HUB_VXLAN" "port1"
        next
    end

    Note

    The virtual wire pair requires an explicit policy to allow traffic between interfaces.

To test the configuration:
  1. Ping the hub FortiGate from the spoke FortiGate:
    user@pc-spoke1:~$ ping 192.168.1.1 -c 3PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.24 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.672 ms
    64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.855 ms
    --- 192.168.1.1 ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2002ms
    rtt min/avg/max/mdev = 0.672/0.923/1.243/0.239 ms
  2. Sniff traffic on the hub FortiGate:
    # diagnose sniffer packet any 'icmp or (udp and port 4789)' 4 0 ainterfaces=[any]
    filters=[icmp or (udp and port 4789)]
    15:00:01.438230 SPOKES in 192.168.255.2.4790 -&gt; 192.168.255.1.4789: udp 106
    <<<<1
    15:00:01.438256 SPOKES_VXLAN in 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    <<<<2
    15:00:01.438260 port1 out 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    <<<<3
    15:00:01.438532 port1 in 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438536 SPOKES_VXLAN out 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438546 SPOKES out 192.168.255.1.4851 -&gt; 192.168.255.2.4789: udp 106

VXLAN over IPsec using a VXLAN tunnel endpoint

This example describes how to implement VXLAN over IPsec VPN using a VXLAN tunnel endpoint (VTEP).

This example shows a specific configuration that uses a hub-and-spoke topology. However, the same logic can be applied to a static VPN with or without XAuth. In this hub-and-spoke topology, dialup VPN is convenient because it uses a single phase 1 dialup definition on the hub FortiGate. Additional spoke tunnels are added without any changes to the hub, other than adding a user account for each additional spoke. Spoke-to-spoke communication is established through the hub. This example assumes the authentication users and user groups have already been created.

IPsec tunnel interfaces are used to support VXLAN tunnel termination. An IP address is set for each tunnel interface. Ping access is allowed for troubleshooting purposes.

VTEPs are created on each of the hub and spokes in order to forward VXLAN traffic through the IPsec tunnels. VXLAN encapsulates OSI layer 2 Ethernet frames within layer 3 IP packets. You will need to either combine the internal port and VXLAN interface into a soft switch, or create a virtual wire pair so that devices behind port1 have direct layer 2 access to remote peers over the VXLAN tunnel. This example uses a switch interface on the hub and a virtual wire pair on the spokes to demonstrate the two different methods.

Finally, in order to apply an IPsec VPN interface on the VXLAN interface setting, net-device must be disabled in the IPsec VPN phase 1 settings. All VXLAN interfaces in this example share the same VXLAN network ID (vni).

To configure the hub FortiGate:
  1. Configure the phase 1 and phase 2 interfaces:
    config vpn ipsec phase1-interface
       edit "SPOKES"
          set type dynamic
          set interface "port2"
          set mode aggressive
          set peertype one
          set net-device disable
          set proposal aes256-sha256
          set xauthtype auto
          set authusrgrp "SPOKES"
          set peerid "SPOKES"
          set psksecret <secret>
       next
    end
    config vpn ipsec phase2-interface
       edit "SPOKES"
          set phase1name "SPOKES"
          set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
       next
    end
  2. Configure the IPsec VPN policy that allows VXLAN traffic between spokes:
    config firewall policy
       edit 1
          set name "VXLAN_SPOKE_to_SPOKE"
          set srcintf "SPOKES"
          set dstintf "SPOKES"
          set srcaddr "NET_192.168.255.0"
          set dstaddr "NET_192.168.255.0"
          set action accept
          set schedule "always"
          set service "UDP_4789"
          set logtraffic all
          set fsso disable
       next
    end
  3. Configure the IPsec tunnel interfaces (the remote IP address is not used, but it is necessary for this configuration):
    config system interface
       edit "SPOKES"
          set vdom "root"
          set ip 192.168.255.1 255.255.255.255
          set allowaccess ping
          set type tunnel
          set remote-ip 192.168.255.254 255.255.255.0
          set snmp-index 12
          set interface "port2"
       next
    end
  4. Configure the VXLAN interface (the remote IP is the tunnel interfaces IPs of the spokes):
    config system VXLAN
       edit "SPOKES_VXLAN"
          set interface "SPOKES"
          set vni 1
          set remote-ip "192.168.255.2" "192.168.255.3"
       next
    end
To configure the spoke FortiGates:
  1. Configure the phase 1 and phase 2 interfaces:
    config vpn ipsec phase1-interface
       edit "HUB"
          set interface "port2"
          set mode aggressive
          set peertype any
          set net-device disable
          set proposal aes256-sha256
          set localid "SPOKES"
          set xauthtype client
          set authusr "SPOKE1"
          set authpasswd <secret>
          set remote-gw <hub public IP>
          set psksecret <secret>
       next
    end
    config vpn ipsec phase2-interface
       edit "HUB"
          set phase1name "HUB"
          set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
          set auto-negotiate enable
          set src-subnet 192.168.255.2 255.255.255.255
       next
    end
    Note

    The hub FortiGate inserts a reverse route pointing to newly established tunnel interfaces for any of the subnets that the spoke FortiGate's source quick mode selectors provides. This is why you should set the tunnel IP address here.

  2. Configure the IPsec VPN policy:
    config firewall policy
       edit 1
          set name "VTEP_IPSEC_POLICY"
          set srcintf "HUB"
          set dstintf "HUB"
          set srcaddr "none"
          set dstaddr "none"
          set action accept
          set schedule "always"
          set service "PING"
          set logtraffic disable
          set fsso disable
       next
    end
  3. Configure the IPsec tunnel interfaces:
    config system interface
       edit "HUB"
          set vdom "root"
          set ip 192.168.255.2 255.255.255.255
          set allowaccess ping
          set type tunnel
          set remote-ip 192.168.255.1 255.255.255.0
          set snmp-index 12
          set interface "port2"
       next
    end
  4. Configure the VXLAN interface (the remote IP is the tunnel interface IP of the hub):
    config system VXLAN
       edit "HUB_VXLAN"
          set interface "HUB"
          set vni 1
          set remote-ip "192.168.255.1"
       next
    end

    To establish a VXLAN tunnel between spokes, you can add a spoke's tunnel IP address in remote-ip.

    Note

    To add more remote IP addresses to a VXLAN interface, the interface cannot be in use. You may want to provision future spokes' remote IP addresses at this point to avoid traffic disruption. Otherwise, you must delete the reference (the policy in this example) before adding remote IP addresses.

To bind the VXLAN interface to the internal interface:
  1. Configure a switch interface on the hub:
    config system switch-interface
        edit "SW"
            set vdom "root"
            set member "port1" "SPOKES_VXLAN"
            set intra-switch-policy {implicit | explicit}
        next
    end

    Note

    Allowing intra-switch traffic is implicitly allowed by default. Use set intra-switch-policy explicit to require firewall policies to allow traffic between switch interfaces.

  2. Configure a virtual wire pair on the spokes:
    config system virtual-wire-pair
        edit "VWP"
            set member "HUB_VXLAN" "port1"
        next
    end

    Note

    The virtual wire pair requires an explicit policy to allow traffic between interfaces.

To test the configuration:
  1. Ping the hub FortiGate from the spoke FortiGate:
    user@pc-spoke1:~$ ping 192.168.1.1 -c 3PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.24 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.672 ms
    64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.855 ms
    --- 192.168.1.1 ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2002ms
    rtt min/avg/max/mdev = 0.672/0.923/1.243/0.239 ms
  2. Sniff traffic on the hub FortiGate:
    # diagnose sniffer packet any 'icmp or (udp and port 4789)' 4 0 ainterfaces=[any]
    filters=[icmp or (udp and port 4789)]
    15:00:01.438230 SPOKES in 192.168.255.2.4790 -&gt; 192.168.255.1.4789: udp 106
    <<<<1
    15:00:01.438256 SPOKES_VXLAN in 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    <<<<2
    15:00:01.438260 port1 out 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    <<<<3
    15:00:01.438532 port1 in 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438536 SPOKES_VXLAN out 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438546 SPOKES out 192.168.255.1.4851 -&gt; 192.168.255.2.4789: udp 106