Fortinet black logo

Administration Guide

VXLAN over IPsec using a VXLAN tunnel endpoint

VXLAN over IPsec using a VXLAN tunnel endpoint

This example describes how to implement VXLAN over IPsec VPN using a VXLAN tunnel endpoint (VTEP).

This example uses a hub and spoke topology. Dialup VPN is used because it allows a single phase 1 dialup definition on the hub FortiGate. Additional spoke tunnels are added with minimal changes to the hub by adding a user account and VXLAN interface for each spoke. Spoke-to-spoke communication is established through the hub. This example assumes that the authentication users and user groups have already been created. While this topology demonstrates hub and spoke with dialup tunnels with XAuth authentication, the same logic can be applied to a static VPN with or without XAuth.

IPsec tunnel interfaces are used to support VXLAN tunnel termination. An IP address is set for each tunnel interface. Ping access is allowed for troubleshooting purposes.

VTEPs are created on the hub and each spoke to forward VXLAN traffic through the IPsec tunnels. VXLAN encapsulates OSI layer 2 Ethernet frames within layer 3 IP packets. You will need to either combine the internal port1 and VXLAN interface into a soft switch, or create a virtual wire pair so that devices behind port1 have direct layer 2 access to remote peers over the VXLAN tunnel. This example uses a switch interface on the hub and a virtual wire pair on the spokes to demonstrate the two different methods.

In order to apply an IPsec VPN interface on the VXLAN interface setting, net-device must be disabled in the IPsec VPN phase 1 settings.

To configure the hub FortiGate:
  1. Configure the IPsec phase 1 interface:

    config vpn ipsec phase1-interface
        edit "SPOKES"
            set type dynamic
            set interface "port2"
            set mode aggressive
            set peertype one
            set net-device disable
            set proposal aes256-sha256
            set xauthtype auto
            set authusrgrp "SPOKES"
            set peerid "SPOKES"
            set psksecret <secret>
        next
    end
    
  2. Configure the IPsec phase 2 interface:

    config vpn ipsec phase2-interface
        edit "SPOKES"
            set phase1name "SPOKES"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
        next
    end
  3. Configure the IPsec VPN policy that allows VXLAN traffic between the spokes:

    config firewall policy
        edit 1
            set name "VXLAN_SPOKE_to_SPOKE"
            set srcintf "SPOKES"
            set dstintf "SPOKES"
            set srcaddr "NET_192.168.255.0"
            set dstaddr "NET_192.168.255.0"
            set action accept
            set schedule "always"
            set service "UDP_4789"
            set logtraffic all
            set fsso disable
        next
    end
  4. Configure the IPsec tunnel interfaces (the remote IP address is not used, but it is necessary for this configuration):

    config system interface
        edit "SPOKES"
            set vdom "root"
            set ip 192.168.255.1 255.255.255.255
            set allowaccess ping
            set type tunnel
            set remote-ip 192.168.255.254 255.255.255.0
            set snmp-index 12
            set interface "port2"
        next
    end
  5. Configure the VXLAN interfaces. Each spoke requires a VXLAN interface with a different VNI. The remote IP is the tunnel interfaces IP of the spokes.

    1. Spoke 1:

      config system VXLAN
          edit "SPOKES_VXLAN1"
              set interface "SPOKES"
              set vni 1
              set remote-ip "192.168.255.2"
          next
      end
    2. Spoke 2:

      config system VXLAN
          edit "SPOKES_VXLAN2"
              set interface "SPOKES"
              set vni 2
              set remote-ip "192.168.255.3"
          next
      end
To configure the spoke FortiGates:
  1. Configure the IPsec phase 1 interface:

    config vpn ipsec phase1-interface
        edit "HUB"
            set interface "port2"
            set mode aggressive
            set peertype any
            set net-device disable
            set proposal aes256-sha256
            set localid "SPOKES"
            set xauthtype client
            set authusr "SPOKE1"
            set authpasswd <secret>
            set remote-gw <hub public IP>
            set psksecret <secret>
        next
    end
  2. Configure the IPsec phase 2 interface:

    config vpn ipsec phase2-interface
        edit "HUB"
            set phase1name "HUB"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
            set auto-negotiate enable
            set src-subnet 192.168.255.2 255.255.255.255
        next
    end
    Note

    The hub FortiGate inserts a reverse route pointing to newly established tunnel interfaces for any of the subnets that the spoke FortiGate's source quick mode selectors provides. This is why you should set the tunnel IP address here.

  3. Configure the IPsec VPN policy:

    config firewall policy
        edit 1
            set name "VTEP_IPSEC_POLICY"
            set srcintf "HUB"
            set dstintf "HUB"
            set srcaddr "none"
            set dstaddr "none"
            set action accept
            set schedule "always"
            set service "PING"
            set logtraffic disable
            set fsso disable
        next
    end
  4. Configure the IPsec tunnel interface:

    config system interface
        edit "HUB"
            set vdom "root"
            set ip 192.168.255.2 255.255.255.255
            set allowaccess ping
            set type tunnel
            set remote-ip 192.168.255.1 255.255.255.0
            set snmp-index 12
            set interface "port2"
        next
    end
  5. Configure the VXLAN interfaces (the remote IP is the tunnel interface IP of the hub):

    1. Spoke 1:

      config system VXLAN
          edit "HUB_VXLAN"
              set interface "HUB"
              set vni 1
              set remote-ip "192.168.255.1"
          next
      end
    2. Spoke 2:

      config system VXLAN
          edit "HUB_VXLAN"
              set interface "HUB"
              set vni 2
              set remote-ip "192.168.255.1"
          next
      end
      
To bind the VXLAN interface to the internal interface:
  1. Configure a switch interface on the hub:
    config system switch-interface
        edit "SW"
            set vdom "root"
            set member "port1" "SPOKES_VXLAN1" "SPOKES_VXLAN2"
            set intra-switch-policy {implicit | explicit}
        next
    end

    Note

    Allowing intra-switch traffic is implicitly allowed by default. Use set intra-switch-policy explicit to require firewall policies to allow traffic between switch interfaces.

  2. Configure a virtual wire pair on the spokes:
    config system virtual-wire-pair
        edit "VWP"
            set member "HUB_VXLAN" "port1"
        next
    end

    Note

    The virtual wire pair requires an explicit policy to allow traffic between interfaces.

To test the configuration:
  1. Ping the hub FortiGate from the spoke FortiGate:
    user@pc-spoke1:~$ ping 192.168.1.1 -c 3
    PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.24 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.672 ms
    64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.855 ms
    --- 192.168.1.1 ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2002 ms
    rtt min/avg/max/mdev = 0.672/0.923/1.243/0.239 ms
  2. Sniff traffic on the hub FortiGate:
    # diagnose sniffer packet any 'icmp or (udp and port 4789)' 4 0 
    interfaces=[any] filters=[icmp or (udp and port 4789)]
    15:00:01.438230 SPOKES in 192.168.255.2.4790 -&gt; 192.168.255.1.4789: udp 106
    15:00:01.438256 SPOKES_VXLAN1 in 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    15:00:01.438260 port1 out 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    15:00:01.438532 port1 in 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438536 SPOKES_VXLAN1 out 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438546 SPOKES out 192.168.255.1.4851 -&gt; 192.168.255.2.4789: udp 106

VXLAN over IPsec using a VXLAN tunnel endpoint

This example describes how to implement VXLAN over IPsec VPN using a VXLAN tunnel endpoint (VTEP).

This example uses a hub and spoke topology. Dialup VPN is used because it allows a single phase 1 dialup definition on the hub FortiGate. Additional spoke tunnels are added with minimal changes to the hub by adding a user account and VXLAN interface for each spoke. Spoke-to-spoke communication is established through the hub. This example assumes that the authentication users and user groups have already been created. While this topology demonstrates hub and spoke with dialup tunnels with XAuth authentication, the same logic can be applied to a static VPN with or without XAuth.

IPsec tunnel interfaces are used to support VXLAN tunnel termination. An IP address is set for each tunnel interface. Ping access is allowed for troubleshooting purposes.

VTEPs are created on the hub and each spoke to forward VXLAN traffic through the IPsec tunnels. VXLAN encapsulates OSI layer 2 Ethernet frames within layer 3 IP packets. You will need to either combine the internal port1 and VXLAN interface into a soft switch, or create a virtual wire pair so that devices behind port1 have direct layer 2 access to remote peers over the VXLAN tunnel. This example uses a switch interface on the hub and a virtual wire pair on the spokes to demonstrate the two different methods.

In order to apply an IPsec VPN interface on the VXLAN interface setting, net-device must be disabled in the IPsec VPN phase 1 settings.

To configure the hub FortiGate:
  1. Configure the IPsec phase 1 interface:

    config vpn ipsec phase1-interface
        edit "SPOKES"
            set type dynamic
            set interface "port2"
            set mode aggressive
            set peertype one
            set net-device disable
            set proposal aes256-sha256
            set xauthtype auto
            set authusrgrp "SPOKES"
            set peerid "SPOKES"
            set psksecret <secret>
        next
    end
    
  2. Configure the IPsec phase 2 interface:

    config vpn ipsec phase2-interface
        edit "SPOKES"
            set phase1name "SPOKES"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
        next
    end
  3. Configure the IPsec VPN policy that allows VXLAN traffic between the spokes:

    config firewall policy
        edit 1
            set name "VXLAN_SPOKE_to_SPOKE"
            set srcintf "SPOKES"
            set dstintf "SPOKES"
            set srcaddr "NET_192.168.255.0"
            set dstaddr "NET_192.168.255.0"
            set action accept
            set schedule "always"
            set service "UDP_4789"
            set logtraffic all
            set fsso disable
        next
    end
  4. Configure the IPsec tunnel interfaces (the remote IP address is not used, but it is necessary for this configuration):

    config system interface
        edit "SPOKES"
            set vdom "root"
            set ip 192.168.255.1 255.255.255.255
            set allowaccess ping
            set type tunnel
            set remote-ip 192.168.255.254 255.255.255.0
            set snmp-index 12
            set interface "port2"
        next
    end
  5. Configure the VXLAN interfaces. Each spoke requires a VXLAN interface with a different VNI. The remote IP is the tunnel interfaces IP of the spokes.

    1. Spoke 1:

      config system VXLAN
          edit "SPOKES_VXLAN1"
              set interface "SPOKES"
              set vni 1
              set remote-ip "192.168.255.2"
          next
      end
    2. Spoke 2:

      config system VXLAN
          edit "SPOKES_VXLAN2"
              set interface "SPOKES"
              set vni 2
              set remote-ip "192.168.255.3"
          next
      end
To configure the spoke FortiGates:
  1. Configure the IPsec phase 1 interface:

    config vpn ipsec phase1-interface
        edit "HUB"
            set interface "port2"
            set mode aggressive
            set peertype any
            set net-device disable
            set proposal aes256-sha256
            set localid "SPOKES"
            set xauthtype client
            set authusr "SPOKE1"
            set authpasswd <secret>
            set remote-gw <hub public IP>
            set psksecret <secret>
        next
    end
  2. Configure the IPsec phase 2 interface:

    config vpn ipsec phase2-interface
        edit "HUB"
            set phase1name "HUB"
            set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
            set auto-negotiate enable
            set src-subnet 192.168.255.2 255.255.255.255
        next
    end
    Note

    The hub FortiGate inserts a reverse route pointing to newly established tunnel interfaces for any of the subnets that the spoke FortiGate's source quick mode selectors provides. This is why you should set the tunnel IP address here.

  3. Configure the IPsec VPN policy:

    config firewall policy
        edit 1
            set name "VTEP_IPSEC_POLICY"
            set srcintf "HUB"
            set dstintf "HUB"
            set srcaddr "none"
            set dstaddr "none"
            set action accept
            set schedule "always"
            set service "PING"
            set logtraffic disable
            set fsso disable
        next
    end
  4. Configure the IPsec tunnel interface:

    config system interface
        edit "HUB"
            set vdom "root"
            set ip 192.168.255.2 255.255.255.255
            set allowaccess ping
            set type tunnel
            set remote-ip 192.168.255.1 255.255.255.0
            set snmp-index 12
            set interface "port2"
        next
    end
  5. Configure the VXLAN interfaces (the remote IP is the tunnel interface IP of the hub):

    1. Spoke 1:

      config system VXLAN
          edit "HUB_VXLAN"
              set interface "HUB"
              set vni 1
              set remote-ip "192.168.255.1"
          next
      end
    2. Spoke 2:

      config system VXLAN
          edit "HUB_VXLAN"
              set interface "HUB"
              set vni 2
              set remote-ip "192.168.255.1"
          next
      end
      
To bind the VXLAN interface to the internal interface:
  1. Configure a switch interface on the hub:
    config system switch-interface
        edit "SW"
            set vdom "root"
            set member "port1" "SPOKES_VXLAN1" "SPOKES_VXLAN2"
            set intra-switch-policy {implicit | explicit}
        next
    end

    Note

    Allowing intra-switch traffic is implicitly allowed by default. Use set intra-switch-policy explicit to require firewall policies to allow traffic between switch interfaces.

  2. Configure a virtual wire pair on the spokes:
    config system virtual-wire-pair
        edit "VWP"
            set member "HUB_VXLAN" "port1"
        next
    end

    Note

    The virtual wire pair requires an explicit policy to allow traffic between interfaces.

To test the configuration:
  1. Ping the hub FortiGate from the spoke FortiGate:
    user@pc-spoke1:~$ ping 192.168.1.1 -c 3
    PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.24 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.672 ms
    64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.855 ms
    --- 192.168.1.1 ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2002 ms
    rtt min/avg/max/mdev = 0.672/0.923/1.243/0.239 ms
  2. Sniff traffic on the hub FortiGate:
    # diagnose sniffer packet any 'icmp or (udp and port 4789)' 4 0 
    interfaces=[any] filters=[icmp or (udp and port 4789)]
    15:00:01.438230 SPOKES in 192.168.255.2.4790 -&gt; 192.168.255.1.4789: udp 106
    15:00:01.438256 SPOKES_VXLAN1 in 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    15:00:01.438260 port1 out 192.168.1.2 -&gt; 192.168.1.1: icmp: echo request
    15:00:01.438532 port1 in 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438536 SPOKES_VXLAN1 out 192.168.1.1 -&gt; 192.168.1.2: icmp: echo reply
    15:00:01.438546 SPOKES out 192.168.255.1.4851 -&gt; 192.168.255.2.4789: udp 106