SD-WAN application monitor using FortiMonitor

The agent-based health check detection mode works with FortiMonitor to provide more accurate user level performance statistics. FortiMonitor acts as an agent and sends health check probes on behalf of the monitored FortiGate interface. FortiMonitor mimics a real user, and the probes return a more accurate application level performance. The SLA information collected from FortiMonitor is sent back to the FortiGate as the monitored interface's SLA information. These statistics can be used to gain a deeper insight into the SD-WAN traffic performance.

FortiGate can log statistics when using FortiMonitor to detect advanced SD-WAN application performance metrics. These logs may also be sent to FortiAnalyzer and FortiManager for review and reporting.

config system sdwan
    config health-check
        edit <name>
            set detect-mode agent-based
        next
    end
    config service
        edit <id>
            set agent-exclusive {enable | disable}
        next
    end
    set app-perf-log-period <time in seconds>
end

The following diagnostic commands can be used to view agent related metrics:

# diagnose sys link-monitor-passive agent <option>

list	List all the collected reports.
list-app	List the details of each application.
flush	Flush all the collected reports.
flush-app	Flush the details of all the applications.
agent-oif-map	List the agent and interface maps.

Example

In this example, routing is achieved through SD-WAN rules. The agent-based health check detection mode creates the FortiMonitor IP address and FortiGate SD-WAN interface map.

This example assumes that the FortiMonitor has already been added to the Security Fabric (see Configuring FortiMonitor for detailed instructions). The FortiMonitor OnSight (client) can be configured for two or more IP addresses, and each IP address is capable of sending application probes to user-specified applications.

Specific routing is implemented on the FortiGate to ensure each FortiMonitor client collects performance statistics for only one SD-WAN member interface. The FortiMonitor is configured to send application-specific probes to measure that application’s performance on a given SD-WAN member. The FortiGate uses the FortiMonitor performance statistics to determine link quality based on application performance by mapping the health check. The link quality for a given application can then be used to steer the matching application traffic with greater accuracy.

To configure the FortiGate:

1. Configure the address objects for each FortiMonitor client:

   config firewall address
       edit "FMR_OnSight1"
           set subnet 10.2.1.80 255.255.255.255
       next
       edit "MR_OnSight2"
           set subnet 10.2.1.81 255.255.255.255
       next
   end

2. Set the logging frequency:

   config system sdwan
       set status enable
       set app-perf-log-period 60
   end

3. Configure the SD-WAN zone and members:

   config system sdwan
       config zone
           edit "virtual-wan-link"
           next
       end
       config members
           edit 1
               set interface "v1236"
               set gateway 10.12.36.2
           next
           edit 2
               set interface "v1237"
               set gateway 10.12.37.20
           next
       end
   end

4. Configure the SD-WAN rules to ensure each OnSight client uses only one SD-WAN member, and map the FortiMonitor IP to an SD-WAN member (interface):

   config system sdwan
       config service
           edit 1
               set dst "all"
               set src "FMR_OnSight1"
               set priority-members 1
               set agent-exclusive enable
           next
           edit 2
               set dst "all"
               set src "FMR_OnSight2"
               set priority-members 2
               set agent-exclusive enable
           next
       end
   end

5. Configure the SD-WAN health check:

   config health-check
       edit "FMR"
           set detect-mode agent-based
           set members 1 2
           config sla
               edit 1
               next
           end
       next
   end

To verify the SD-WAN member performance:

1. Verify the health check diagnostics:

   # diagnose sys sdwan health-check 
   Health Check(FMR): 
   Seq(1 v1236): state(alive), packet-loss(0.000%) latency(183.214), jitter(0.124), mos(4.225), bandwidth-up(999992), bandwidth-dw(999976), bandwidth-bi(1999968) sla_map=0x0
   Seq(2 v1237): state(alive), packet-loss(0.000%) latency(182.946), jitter(0.100), mos(4.226), bandwidth-up(999998), bandwidth-dw(999993), bandwidth-bi(1999991) sla_map=0x0
   

2. Verify the collected reports:

   # diagnose sys link-monitor-passive agent list 
          v1236( 23) | src=10.2.1.80 | latency=183.2   20:27:24 | jitter=0.1     20:27:24 | pktloss=0.0  % 20:27:24
          v1237( 24) | src=10.2.1.81 | latency=182.9   20:27:24 | jitter=0.1     20:27:24 | pktloss=0.0  % 20:27:24
   

3. Verify the details of each application:

   # diagnose sys link-monitor-passive agent list-app 
   app_id=0x00000000, app=fortinet.com, dev=v1236(23)
           latency=183.2, jitter=0.1, pktloss=0.0,ntt=99.2,srt=384.8,app_err=0.0, 20:28:25
   app_id=0x00000000, app=fortinet.com, dev=v1237(24)
           latency=183.1, jitter=0.5, pktloss=0.0,ntt=104.4,srt=377.8,app_err=0.0, 20:28:25
   

4. Verify the agent and interface maps:

   # diagnose sys link-monitor-passive agent agent-oif-map 
   oif=v1236(23), src=10.2.1.80
   oif=v1237(24), src=10.2.1.81
   

5. Review the SD-WAN logs:

6. # execute log filter category event
   # execute log filter field subtype sdwan
   # execute log display
   
   1: date=2023-01-27 time=16:32:15 eventtime=1674865935918381398 tz="-0800" logid="0113022937" type="event" subtype="sdwan" level="information" vd="root" logdesc="Virtuan WAN Link application performance metrics via FortiMonitor" eventtype="Application Performance Metrics" app="fortinet.com" appid=0 interface="v1237" latency="200.2" jitter="0.6" packetloss="0.0" serverresponsetime="827.7" networktransfertime="107.7" apperror="0.0" timestamp="01-28 00:31:59" msg="Application Performance Metrics via FortiMonitor"
   
   2: date=2023-01-27 time=16:32:15 eventtime=1674865935918367770 tz="-0800" logid="0113022937" type="event" subtype="sdwan" level="information" vd="root" logdesc="Virtuan WAN Link application performance metrics via FortiMonitor" eventtype="Application Performance Metrics" app="fortinet.com" appid=0 interface="v1236" latency="200.0" jitter="0.3" packetloss="0.0" serverresponsetime="870.6" networktransfertime="130.4" apperror="0.0" timestamp="01-28 00:31:59" msg="Application Performance Metrics via FortiMonitor"
   
   3: date=2023-01-27 time=16:31:15 eventtime=1674865875917685437 tz="-0800" logid="0113022937" type="event" subtype="sdwan" level="information" vd="root" logdesc="Virtuan WAN Link application performance metrics via FortiMonitor" eventtype="Application Performance Metrics" app="fortinet.com" appid=0 interface="v1237" latency="200.5" jitter="0.7" packetloss="0.0" serverresponsetime="1008.9" networktransfertime="129.8" apperror="0.0" timestamp="01-28 00:31:02" msg="Application Performance Metrics via FortiMonitor"
   
   4: date=2023-01-27 time=16:31:15 eventtime=1674865875917672824 tz="-0800" logid="0113022937" type="event" subtype="sdwan" level="information" vd="root" logdesc="Virtuan WAN Link application performance metrics via FortiMonitor" eventtype="Application Performance Metrics" app="fortinet.com" appid=0 interface="v1236" latency="200.3" jitter="0.8" packetloss="0.0" serverresponsetime="825.4" networktransfertime="106.4" apperror="0.0" timestamp="01-28 00:31:02" msg="Application Performance Metrics via FortiMonitor"
   
   5: date=2023-01-27 time=16:30:15 eventtime=1674865815912801725 tz="-0800" logid="0113022937" type="event" subtype="sdwan" level="information" vd="root" logdesc="Virtuan WAN Link application performance metrics via FortiMonitor" eventtype="Application Performance Metrics" app="fortinet.com" appid=0 interface="v1237" latency="200.1" jitter="0.4" packetloss="0.0" serverresponsetime="845.4" networktransfertime="116.0" apperror="0.0" timestamp="01-28 00:30:01" msg="Application Performance Metrics via FortiMonitor"
   
   6: date=2023-01-27 time=16:30:15 eventtime=1674865815912786458 tz="-0800" logid="0113022937" type="event" subtype="sdwan" level="information" vd="root" logdesc="Virtuan WAN Link application performance metrics via FortiMonitor" eventtype="Application Performance Metrics" app="fortinet.com" appid=0 interface="v1236" latency="200.0" jitter="0.3" packetloss="0.0" serverresponsetime="1032.0" networktransfertime="138.9" apperror="0.0" timestamp="01-28 00:30:01" msg="Application Performance Metrics via FortiMonitor"