Application performance monitoring NEW

FortiOS includes application performance monitoring (APM), which passively monitors common TCP metrics for each application and calculates application-level network performance metrics over multiple traffic sessions.

By leveraging these APM metrics, administrators can:

• Pinpoint latency and find issues quicker and proactively.

• Troubleshoot application performance through detailed network metrics.

• Simplify configuration and management of applications.

• Monitor trends over time of business-critical end user applications within your FortiGate LAN environment.

Prerequisites

• APM requires the SD-WAN Underlay and Application Monitoring license, which relies on the FortiGuard SLA database to identify popular SaaS and Internet destinations in monitored sessions.

• Passive health check must be configured in SD-WAN settings for APM to work.

Considerations

• When app-monitor is enabled in a firewall policy, NPU offloading for the firewall policy is automatically disabled.

• You can display TCP metrics using the diagnose sys session list command, or you can view traffic logs in the CLI or GUI.

• SD-WAN traffic steering remains independent from the measured TCP session metrics.

The new metrics include:

Metric	Description
Server Response Time	Latency between the time a packet is sent from the origin and the time a response is sent back from the destination (milliseconds).
Network Transfer Time	Latency between the time a packet is sent to the origin and the time to get a response from the origin (milliseconds). Also known as TCP connection time.
Latency	Overall latency between origin and destination (milliseconds).
RTT Sample	Number of sessions used to determine server response time, network transfer time, and latency metrics.
Origin Jitter	Jitter at the origin (milliseconds).
Reply Jitter	Jitter at the destination (milliseconds).
Jitter	Jitter between origin and destination (milliseconds).
Origin Packet Loss	Packet loss at the origin (percentage).
Reply Packet Loss	Packet loss at the destination (percentage).
Packet Loss	Overall packet loss between origin and destination (percentage).
Retransmission Sample	Number of sessions used to determine packet loss and retransmission metrics.
Origin Retransmission	Total number of retransmits that occurred in the origin to the destination traffic flow.
Reply Retransmission	Total number of retransmits that occurred in the destination to the origin traffic flow.
SYN Retransmission	Total number of SYN retransmits that occurred from the origin.
SYN-ACK Retransmission	Total number of SYN-ACK retransmits that occurred from the origin.
Origin Reset	Total number of origin resets that occurred.
Reply Reset	Total number of destination resets that occurred.

The main network metrics used to quantify application-level performance are latency, jitter, and packet loss.

• Latency is the delay in seconds for packets to travel across the network, measured in milliseconds.

• Jitter is the variation in delay in seconds for packets to travel across the network, measured in milliseconds.

• Packet loss is the percentage of packets from the origin that fail to arrive at the destination when attempting to travel across the network.

To enable passive monitoring of applications:

config firewall policy
    edit <entry>
        set app-monitor enable
        set passive-wan-health-measurement enable 
    next
end

Example

In this example, SD-WAN is configured with a zone named virtual-wan-link, and it contains two members (vlan100 and vd1-p1). A firewall policy is configured for the SD-WAN zonemetrics with application performance monitoring from the PC to a server.

To configure SD-WAN:

1. Configure SD-WAN:

   config system sdwan
       set status enable
       config zone
           edit "virtual-wan-link"
           next
       end
       config members
           edit 1
               set interface "vd1-p1"
           next
           edit 2
               set interface "vlan100"
               set gateway 172.16.206.2
           next
       end
       config health-check
           edit "1"
               set detect-mode passive
               set members 0
           next
       end
       config service
           edit 1
               set name "1"
               set dst "all"
               set src "172.16.205.0"
               set priority-members 1 2
           next
       end
   end

2. Identify the preferred interface:

   In this example vd1-p1 is the preferred SD-WAN member.

   # diagnose sys sdwan service4
   
   Service(1): Address Mode(IPV4) flags=0x4200 use-shortcut-sla use-shortcut
    Tie break: cfg
    Shortcut priority: 2
     Gen(1), TOS(0x0/0x0), Protocol(0): src(1->65535):dst(1->65535), Mode(manual)
     Members(2):
       1: Seq_num(1 vd1-p1 virtual-wan-link), alive, selected          
       2: Seq_num(2 vlan100 virtual-wan-link), alive, selected
     Src address(1):
           172.16.205.0-172.16.205.255
     Dst address(1):
           0.0.0.0-255.255.255.255

3. Configure a firewall policy for the SD-WAN zone to monitor traffic from the PC:

   In this example, the dstintf option is set to the SD-WAN zone (virtual-wan-link), the srcaddr option identifies the PC (172.16.205.0), and application performance monitoring is enabled.

   config firewall policy
       edit 1
           set name "APM"
           set srcintf "any"
           set dstintf "virtual-wan-link"
           set action accept
           set srcaddr "172.16.205.0"                        
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
           set app-monitor enable 
           set passive-wan-health-measurement enable 
           set utm-status enable
           set ssl-ssh-profile "certificate-inspection"
           set application-list "g-default"
           set logtraffic all                                          
           set auto-asic-offload disable
       next
   end

4. As traffic passes from the PC through FortiGate to the server, TCP traffic is measured and logged, and you can view a session list:

   # diagnose sys session list
   
   session info: proto=6 proto_state=11 duration=172 expire=3577 timeout=3600 refresh_dir=both flags=00000000 socktype=0 sockport=0 av_idx=0 use=3
   origin-shaper=
   reply-shaper=
   per_ip_shaper=
   class_id=0 ha_id=0 policy_dir=0 tunnel=/ tun_id=172.16.209.2/0.0.0.0 vlan_cos=0/255
   state=log may_dirty f00 f02 app_valid
   statistic(bytes/packets/allow_err): org=59961/864/1 reply=2663311/2103/1 tuples=2
   tx speed(Bps/kbps): 2/0 rx speed(Bps/kbps): 3/0
   orgin->sink: org pre->post, reply pre->post dev=15->115/115->15 gwy=172.16.209.2/172.16.205.100
   hook=pre dir=org act=noop 172.16.205.100:51128->172.16.202.2:22(0.0.0.0:0)
   hook=post dir=reply act=noop 172.16.202.2:22->172.16.205.100:51128(0.0.0.0:0)
   pos/(before,after) 0/(0,0), 0/(0,0)
   misc=0 policy_id=1 pol_uuid_idx=843 auth_info=0 chk_client_info=0 vd=0
   serial=00006eb8 tos=ff/ff app_list=6000 app=16060 url_cat=0
   sdwan_mbr_seq=1 sdwan_service_id=1
   rpdb_link_id=ff000001 ngfwid=n/a
   tcp_srt=240 tcp_nrt=0 tcp_org_rtrs=17 tcp_rpl_rtrs=273 tcp_syn_rtrs=0 tcp_syn_ack_rtrs=0 tcp_rst=00
   npu_state=0x1041001 no_offload
   no_ofld_reason:  disabled-by-policy non-npu-intf
   total session: 1

5. View detailed application performance metrics in SD-WAN logs:

   # execute log display
   
   1: date=2025-03-06 time=09:40:33 eventtime=1741210833244790449 tz="+1200" logid="0113022941" type="event" subtype="sdwan" level="information" vd="root" logdesc="SDWAN application performance metrics via kernel" eventtype="Application Performance Metrics" appid=16091 interface="vd1-p1" serverresponsetime="162.0" networktransfertime="0.0" latency="162.0" rttsample=5 originjitter="0" replyjitter="100" jitter="100.0" originpktloss="21.8" replypktloss="2.6" packetloss="3.7" retransample=6 originretransmission=13 replyretransmission=17 synretransmission=0 synackretransmission=0 originreset=0 replyreset=0 msg="Application Performance Metrics via kernel"

   To interpret this log: the application 16091 - Telnet is experiencing a latency of 162 ms on 5 session samples. Some packet losses were experienced in both origin and reply directions, leading to some retransmissions. Details are listed in the table.

   Metric	Value
   Server Response Time (ms)	162.0
   Network Transfer Time (ms)	0.0
   Latency (ms)	162.0
   RTT Sample	5
   Origin Jitter (ms)	0
   Reply Jitter (ms)	100
   Jitter (ms)	100.0
   Origin Packet Loss (%)	21.8
   Reply Packet Loss (%)	2.6
   Packet Loss (%)	3.7
   Retransmission Sample	6
   Origin Retransmission	13
   Reply Retransmission	17
   SYN Retransmission	0
   SYN-ACK Retransmission	0
   Origin Reset	0
   Reply Reset	0