Version:

Version:

Version:

Version:

Version:

Version:

Version:

Version:

Version:

Version:

Version:

Version:


Table of Contents

Administration Guide

    Diagnosing debug flow

Debugging traffic flow at user level with diagnose commands

The most commonly used diagnose debug flow commands are combined as below:

Reset enabled diagnose settings, turn on debug log output with timestamp

diagnose debug reset

diagnose debug enable

diagnose debug timestamp enable

 

Add filters and start the flow trace

diagnose debug flow filter flow-detail 7  #Enables messages from each packet processing module and packet flow traces

diagnose debug flow filter http-detail 7 #HTTP parser details

diagnose debug flow filter module-detail status on  #Turn on details from modules processing the flow

diagnose debug flow filter server-ip 192.168.12.12 #The VIP in RP mode or the real server IP in TP/TI mode

diagnose debug flow filter client-ip 192.168.12.1  #The client IP

diagnose debug flow trace start

 

To stop output

diagnose debug flow trace stop

Diagnose debug disable

Debugging traffic flow at kernel level

Change the debug levels in the backend settings, then kernel level debug logs will be recorded in dmesg. This method is useful to track traffic flow processing in the system kernel.

1) /proc/tproxy/debug    # for transparent mode.

  • echo "FFFF F" > proc/tproxy/debug: output logs to dmesg with a detailed level

  • echo "XXXX F" > proc/tproxy/debug: don’t forget to turn off debug logs

Use the same way to turn on debug logs for reverse-proxy and wccp mode.

Some details:

/var/log# more /proc/tproxy/debug

Debug modules : HOOK4 HOOK6 HASH POLICY

        HOOK4 : for netfilter hook ipv4

        HOOK6 : for netfilter hook ipv6

        HASH : for tproxy hash

        POLICY : for policy management

        FFFF : for all above

        XXXX : cleanup all above

        PASS : for bypass this module in kernel path

        LOIP : for enable / disable local ip filter in hook4

        PIP : <PIP [1,0] ip> for only enbale this ip upto proxyd

Debug levels : 1 2 4 8

        1 : for error message

        2 : for data packet info

        4 : for data following info

        8 : for function entry/exit info

Current debug info : FFFF 15, mbypass = 0, sysmode : 2, localip : 0, proxyd-ip : 0.0.0.0

        ex : echo "HOOK4 F" > debug > debug

        ex : echo "PIP 1 10.200.2.1" > debug

Example:

[BEGIN] 9/13/2021 23:35:55

/# dmesg

[553897.203831] (tproxy) (/Chroot_Build/34/SVN_REPO_CHILD/FortiWEB/kernel/modules/tproxy/tproxy_policy.c:433) get vserver(240.0.0.29), vport(9781), dir(1)

[553897.203834] (tproxy) ====> get vserver(240.0.0.29), vport(9781), mark(1835264/1835264), incoming (vzone_p3p4_vlan) tcp info : src:(192.168.11.1:48310), dst:(192.168.11.2:80)

[553897.203836] (tproxy) (465) incoming (vzone_p3p4_vlan) tcp info : src:(192.168.11.1:48310), dst:(192.168.11.2:80) -ipid(63355) iptlen(60) seq(2348868809) ack_seq(0) syn(1) ack(0) fin(0) rst(0) psh(0)

[553897.203838] (tproxy) [fortiweb-tproxy] redirecting: proto 6 192.168.11.2:80 -> 240.0.0.29:9781, ipid(63355) iplen(60) mark: 1c0100

[553897.203855] (tproxy)

[553897.203855]

[553897.203855] ====> out to client : src:(192.168.11.2:80), dst:(192.168.11.1:48310)- seq(1319007036) ack_seq(2348868810) syn(1) ack(1) fin(0) rst(0) psh(0)

[553897.203856] (tproxy) [POST_ROUTING]: TO CLIENT OK,  192.168.11.2:80->192.168.11.1:48310, todevname:port3vlan101, flag 4000

2) /proc/rptproxy/debug    #for reverse-proxy mode

/var/log# more /proc/rptproxy/debug

Debug modules : HOOK4 HOOK6 HASH POLICY

        HOOK4 : for netfilter hook ipv4

        HOOK6 : for netfilter hook ipv6

        POLICY : for policy management

        FFFF : for all above

        XXXX : cleanup all above

        PASS : for bypass this module in kernel path

        LOIP : for enable / disable local ip filter in hook4

        PIP : <PIP [1,0] ip> for only enbale this ip upto proxyd

Debug levels : 1 2 4 8

        ...

Current debug info :  0, mbypass = 0, sysmode : 2, localip : 0, proxyd-ip : 0.0.0.0

3) /proc/wproxy/debug    #for wccp mode

/var/log# more /proc/wproxy/debug

Debug modules : HOOK4 HOOK6 POLICY

        HOOK4 : for netfilter hook ipv4

        HOOK6 : for netfilter hook ipv4

        POLICY : for policy management

        FFFF : for all above

        XXXX : cleanup all above

        PASS : for bypass this module in kernel path

Debug levels : 1 2 4 8

        ...

Current debug info :  0, mbypass = 0, sysmode : 1

How to capture network packets in FortiWeb

Capturing network packets is a useful and direct method when troubleshooting network issues, including TCP connection establishment issues, SSL handshake issues or analyzing HTTP issues.

Usually it’s better to enable diagnose debug flow and capture packets at the same time, then analyze them together.

    Diagnosing debug flow

Debugging traffic flow at user level with diagnose commands

The most commonly used diagnose debug flow commands are combined as below:

Reset enabled diagnose settings, turn on debug log output with timestamp

diagnose debug reset

diagnose debug enable

diagnose debug timestamp enable

 

Add filters and start the flow trace

diagnose debug flow filter flow-detail 7  #Enables messages from each packet processing module and packet flow traces

diagnose debug flow filter http-detail 7 #HTTP parser details

diagnose debug flow filter module-detail status on  #Turn on details from modules processing the flow

diagnose debug flow filter server-ip 192.168.12.12 #The VIP in RP mode or the real server IP in TP/TI mode

diagnose debug flow filter client-ip 192.168.12.1  #The client IP

diagnose debug flow trace start

 

To stop output

diagnose debug flow trace stop

Diagnose debug disable

Debugging traffic flow at kernel level

Change the debug levels in the backend settings, then kernel level debug logs will be recorded in dmesg. This method is useful to track traffic flow processing in the system kernel.

1) /proc/tproxy/debug    # for transparent mode.

  • echo "FFFF F" > proc/tproxy/debug: output logs to dmesg with a detailed level

  • echo "XXXX F" > proc/tproxy/debug: don’t forget to turn off debug logs

Use the same way to turn on debug logs for reverse-proxy and wccp mode.

Some details:

/var/log# more /proc/tproxy/debug

Debug modules : HOOK4 HOOK6 HASH POLICY

        HOOK4 : for netfilter hook ipv4

        HOOK6 : for netfilter hook ipv6

        HASH : for tproxy hash

        POLICY : for policy management

        FFFF : for all above

        XXXX : cleanup all above

        PASS : for bypass this module in kernel path

        LOIP : for enable / disable local ip filter in hook4

        PIP : <PIP [1,0] ip> for only enbale this ip upto proxyd

Debug levels : 1 2 4 8

        1 : for error message

        2 : for data packet info

        4 : for data following info

        8 : for function entry/exit info

Current debug info : FFFF 15, mbypass = 0, sysmode : 2, localip : 0, proxyd-ip : 0.0.0.0

        ex : echo "HOOK4 F" > debug > debug

        ex : echo "PIP 1 10.200.2.1" > debug

Example:

[BEGIN] 9/13/2021 23:35:55

/# dmesg

[553897.203831] (tproxy) (/Chroot_Build/34/SVN_REPO_CHILD/FortiWEB/kernel/modules/tproxy/tproxy_policy.c:433) get vserver(240.0.0.29), vport(9781), dir(1)

[553897.203834] (tproxy) ====> get vserver(240.0.0.29), vport(9781), mark(1835264/1835264), incoming (vzone_p3p4_vlan) tcp info : src:(192.168.11.1:48310), dst:(192.168.11.2:80)

[553897.203836] (tproxy) (465) incoming (vzone_p3p4_vlan) tcp info : src:(192.168.11.1:48310), dst:(192.168.11.2:80) -ipid(63355) iptlen(60) seq(2348868809) ack_seq(0) syn(1) ack(0) fin(0) rst(0) psh(0)

[553897.203838] (tproxy) [fortiweb-tproxy] redirecting: proto 6 192.168.11.2:80 -> 240.0.0.29:9781, ipid(63355) iplen(60) mark: 1c0100

[553897.203855] (tproxy)

[553897.203855]

[553897.203855] ====> out to client : src:(192.168.11.2:80), dst:(192.168.11.1:48310)- seq(1319007036) ack_seq(2348868810) syn(1) ack(1) fin(0) rst(0) psh(0)

[553897.203856] (tproxy) [POST_ROUTING]: TO CLIENT OK,  192.168.11.2:80->192.168.11.1:48310, todevname:port3vlan101, flag 4000

2) /proc/rptproxy/debug    #for reverse-proxy mode

/var/log# more /proc/rptproxy/debug

Debug modules : HOOK4 HOOK6 HASH POLICY

        HOOK4 : for netfilter hook ipv4

        HOOK6 : for netfilter hook ipv6

        POLICY : for policy management

        FFFF : for all above

        XXXX : cleanup all above

        PASS : for bypass this module in kernel path

        LOIP : for enable / disable local ip filter in hook4

        PIP : <PIP [1,0] ip> for only enbale this ip upto proxyd

Debug levels : 1 2 4 8

        ...

Current debug info :  0, mbypass = 0, sysmode : 2, localip : 0, proxyd-ip : 0.0.0.0

3) /proc/wproxy/debug    #for wccp mode

/var/log# more /proc/wproxy/debug

Debug modules : HOOK4 HOOK6 POLICY

        HOOK4 : for netfilter hook ipv4

        HOOK6 : for netfilter hook ipv4

        POLICY : for policy management

        FFFF : for all above

        XXXX : cleanup all above

        PASS : for bypass this module in kernel path

Debug levels : 1 2 4 8

        ...

Current debug info :  0, mbypass = 0, sysmode : 1

How to capture network packets in FortiWeb

Capturing network packets is a useful and direct method when troubleshooting network issues, including TCP connection establishment issues, SSL handshake issues or analyzing HTTP issues.

Usually it’s better to enable diagnose debug flow and capture packets at the same time, then analyze them together.