Controlling return path with auxiliary session
When multiple incoming or outgoing interfaces are used in ECMP or for load balancing, changes to routing, incoming, or return traffic interfaces impacts how an existing sessions handles the traffic. Auxiliary sessions can be used to handle these changes to traffic patterns.
|
|
To enable or disable the auxiliary session feature:
config system settings
set auxiliary-session {enable | disable*}
end
|
|
When enabling auxiliary sessions, consider the impact of routing in both traffic directions. In topologies such as SD-WAN hub and spoke or ADVPN deployments, the symmetry of the return traffic is important for maintaining the stability of the session. It is expected that the spoke selects the outbound interface and path, and the other nodes obey and reply symmetrically. It is recommended to disable auxiliary in these scenarios, and others where incoming and return traffic symmetry is expected. |
Scenarios
Incoming traffic is from the client to the server. Return traffic is from the server to the client.
Scenario 1 - Return traffic returns on the original outgoing interface
In this scenario, a session is established between port1 and port3. When the return traffic hits port3:
Auxiliary sessions disabled:
The reply to the client egresses on the original incoming interface, port1. If policy routes or SD-WAN rules are configured, the next hop gateway is applied if the output device is the same as the original incoming interface.
Auxiliary sessions enabled:
The reply to the client egresses on the best route in the routing table:
-
If the best route is port1, then it will egress on port1.
-
If the best route is port2, then it will egress on port2.
If policy routes or SD-WAN rules are configured, they must be matched to determine the egress interface. If both are configured, policy routes have higher priority.
Scenario 2 - Return traffic returns on an interfaces other than the original outgoing interfaces
In this scenario, a session is established between port1 and port3. When the return traffic hits port4:
Auxiliary sessions disabled:
-
The session is dirtied and then gets refreshed, and interfaces on the session are updated. This continuous state change to dirty prevents the session from being offloaded.
-
If there is a high traffic volume or flapping between the interfaces, the CPU usage increases.
Auxiliary sessions enabled:
An auxiliary session is created for the existing session, and traffic returns to the client as normal on the auxiliary session.
Scenario 3 - Incoming traffic enters on an interfaces other than the original incoming interfaces
In this scenario, a session is established between port1 and port3. When the incoming traffic hits port2:
Auxiliary sessions disabled:
The session is dirtied and then gets refreshed, and interfaces on the session are updated. This continuous state change to dirty prevents the session from being offloaded.
Auxiliary sessions enabled:
An auxiliary session is created for the existing session, and traffic is forwarded to the server as normal on the auxiliary session.
Scenario 4 - the routing table is changed
In this scenario, a session has been established between port1 and port3, when a new route on port4 is updated as the route to the server.
Auxiliary sessions disabled:
As long as there is a route to the destination, the session will not be dirtied or refreshed. Even though there is a better route, traffic continues on the original path between port1 and port3.
Auxiliary sessions enabled:
The session is dirtied and then gets refreshed, and interfaces on the session are updated. This continuous state change to dirty prevents the session from being offloaded.
Effect on NPU offloading sessions
When the auxiliary session feature is disabled, there is always one session. If the incoming or return interface changes, the FortiGate marks the session as dirty and updates the session's interfaces. This cannot be done by the NPU, so the session is not offloaded to the NPU, and is processed by the CPU instead. If Equal-Cost Multi-Path (ECMP) causes the interface to keep changing, then it will use significant CPU resources.
When the auxiliary session feature is enabled and the incoming or return interface changes, it creates an auxiliary session, and all traffic can continue to be processed by the NPU.
Verification
When an auxiliary, or reflect, session is created, it will appear as a reflect session below the existing session:
# diagnose sys session list session info: proto=17 proto_state=00 duration=111 expire=175 timeout=0 flags=00000000 socktype=0 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255 state=may_dirty npu statistic(bytes/packets/allow_err): org=131/4/1 reply=0/0/0 tuples=2 tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0 orgin->sink: org pre->post, reply pre->post dev=36->38/38->36 gwy=10.1.2.3/0.0.0.0 hook=pre dir=org act=noop 10.1.100.22:51926->172.16.204.44:5001(0.0.0.0:0) hook=post dir=reply act=noop 172.16.204.44:5001->10.1.100.22:51926(0.0.0.0:0) src_mac=90:6c:ac:19:19:58 misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=2 serial=00002b11 tos=ff/ff app_list=0 app=0 url_cat=0 sdwan_mbr_seq=0 sdwan_service_id=0 rpdb_link_id=00000000 rpdb_svc_id=0 ngfwid=n/a npu_state=0x000400 npu info: flag=0x91/0x00, offload=8/0, ips_offload=0/0, epid=129/0, ipid=142/0, vlan=0x0016/0x0000 vlifid=142/0, vtag_in=0x0016/0x0000 in_npu=1/0, out_npu=1/0, fwd_en=0/0, qid=4/0 no_ofld_reason: reflect info 0: dev=37->38/38->37 npu_state=0x000400 npu info: flag=0x91/0x00, offload=8/0, ips_offload=0/0, epid=129/0, ipid=142/0, vlan=0x0017/0x0000 vlifid=142/0, vtag_in=0x0017/0x0000 in_npu=1/0, out_npu=1/0, fwd_en=0/0, qid=4/0 total reflect session num: 1 total session 1
When a session is dirtied, a dirty flag is added to it:
# diagnose sys session list session info: proto=17 proto_state=00 duration=28 expire=152 timeout=0 flags=00000000 socktype=0 sockport=0 av_idx=0 use=3 origin-shaper= reply-shaper= per_ip_shaper= class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255 state=dirty may_dirty npu statistic(bytes/packets/allow_err): org=68/2/1 reply=0/0/0 tuples=2 tx speed(Bps/kbps): 2/0 rx speed(Bps/kbps): 0/0 orgin->sink: org pre->post, reply pre->post dev=0->0/0->0 gwy=0.0.0.0/0.0.0.0 hook=pre dir=org act=noop 10.1.100.22:51926->172.16.204.44:5001(0.0.0.0:0) hook=post dir=reply act=noop 172.16.204.44:5001->10.1.100.22:51926(0.0.0.0:0) src_mac=90:6c:ac:19:19:58 dst_mac=02:6c:ac:5c:c6:f9 misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=2 serial=00002b2c tos=ff/ff app_list=0 app=0 url_cat=0 sdwan_mbr_seq=0 sdwan_service_id=0 rpdb_link_id=00000000 rpdb_svc_id=0 ngfwid=n/a npu_state=0x000400 npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000 vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0 no_ofld_reason: total session 1
When an auxiliary session is created, NPU offloading will continue in the reflect session:
# diagnose sys session list session info: proto=17 proto_state=01 duration=169 expire=129 timeout=0 flags=00000000 socktype=0 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255 state=may_dirty npu statistic(bytes/packets/allow_err): org=131/4/1 reply=66/2/1 tuples=2 tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0 orgin->sink: org pre->post, reply pre->post dev=36->38/38->36 gwy=10.1.2.3/172.17.2.1 hook=pre dir=org act=noop 10.1.100.22:51926->172.16.204.44:5001(0.0.0.0:0) hook=post dir=reply act=noop 172.16.204.44:5001->10.1.100.22:51926(0.0.0.0:0) src_mac=90:6c:ac:19:19:58 misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=2 serial=00002b11 tos=ff/ff app_list=0 app=0 url_cat=0 sdwan_mbr_seq=0 sdwan_service_id=0 rpdb_link_id=00000000 rpdb_svc_id=0 ngfwid=n/a npu_state=0x000c00 npu info: flag=0x91/0x81, offload=8/8, ips_offload=0/0, epid=129/142, ipid=142/128, vlan=0x0016/0x0016 vlifid=142/128, vtag_in=0x0016/0x0016 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=4/4 reflect info 0: dev=37->38/38->37 npu_state=0x000400 npu info: flag=0x91/0x00, offload=8/0, ips_offload=0/0, epid=129/0, ipid=142/0, vlan=0x0017/0x0000 vlifid=142/0, vtag_in=0x0017/0x0000 in_npu=1/0, out_npu=1/0, fwd_en=0/0, qid=4/0 total reflect session num: 1 total session 1