Source port exhaustion for webproxy-forward-server

For large-scale setup with potential large number of concurrent requests from clients, the proxy server might encounter source port exhaustion, which means all the available source ports within the default port range 1024-65501 are used up and requests can no longer come through. You can configure the web proxy in the following ways to fix a source port exhaustion issue or to avoid potential source port exhaustion.

Starting from 7.2.3, you can use the following logs to learn about source port usage: High source port usage—This log is recorded when more than half of the available source ports on an IP is in use during the last few consecutive attempts of the FortiProxy to get a source port. Source port exhaustion—This log is recorded when no available source port can be found for a source IP.

• Add explicit outgoing IPs for the forwarding server and configure the interface accordingly using the following commands. Doing so multiplies the number of available source ports by the number of explicit outgoing IPs. In the command excerpt below, two outgoing IPs are configured, which results in twice the number of available source ports for the forwarding server.

  FPX31 # show web-proxy global

  config web-proxy global

  set proxy-fqdn "default.fqdn"

  set explicit-outgoing-ip 3.3.3.3 4.4.4.4

  end

  FPX31 # show system interface port2

  config system interface

  edit "port2"

  set ip 10.10.1.31 255.255.255.0

  set allowaccess ping https ssh snmp http telnet fgfm radius-acct probe-response fabric ftm speed-test

  set type physical

  set explicit-web-proxy enable

  set snmp-index 2

  set secondary-IP enable

  config ipv6

  set ip6-send-adv enable

  set ip6-other-flag enable

  end

  config secondaryip

  edit 1

  set ip 3.3.3.3 255.255.255.255

  next

  edit 2

  set ip 4.4.4.4 255.255.255.255

  next

  FPX31 # dia sniffer packet any 'tcp[13] & 2 != 0'

  interfaces=[any]

  filters=[tcp[13] & 2 != 0]

  1.038768 10.10.1.22.45027 -> 10.10.1.31.8080: syn 2399357371

  1.038800 10.10.1.31.8080 -> 10.10.1.22.45027: syn 223499737 ack 2399357372

  1.039199 3.3.3.3.19200 -> 10.10.1.32.8080: syn 4162585524

  1.039302 10.10.1.32.8080 -> 3.3.3.3.19200: syn 2987164399 ack 4162585525

  1.059679 10.10.1.23.52115 -> 10.10.1.31.8080: syn 252536669

  1.059716 10.10.1.31.8080 -> 10.10.1.23.52115: syn 2905875968 ack 252536670

  1.060107 4.4.4.4.18796 -> 10.10.1.32.8080: syn 2922300884

  1.060246 10.10.1.32.8080 -> 4.4.4.4.18796: syn 1200833986 ack 2922300885

  1.079254 10.10.1.24.49025 -> 10.10.1.31.8080: syn 1302803618

  1.079288 10.10.1.31.8080 -> 10.10.1.24.49025: syn 1596008331 ack 1302803619

  1.079700 4.4.4.4.18798 -> 10.10.1.32.8080: syn 3769624749

  1.079803 10.10.1.32.8080 -> 4.4.4.4.18798: syn 3544239867 ack 3769624750

  1.098888 10.10.1.2.44602 -> 10.10.1.31.8080: syn 1827828738

  1.098920 10.10.1.31.8080 -> 10.10.1.2.44602: syn 2044038030 ack 1827828739

  1.099345 3.3.3.3.19206 -> 10.10.1.32.8080: syn 1398974532

  1.099493 10.10.1.32.8080 -> 3.3.3.3.19206: syn 3846447541 ack 1398974533

• Use IP pools to configure a range of IPs that can be used, which multiplies the number of available source ports by the number of available IPs in the range. To configure an IP pool in the CLI:

  FPX31 # show firewall ippool

  config firewall ippool

  edit "pool1"

  set startip 1.1.1.100

  set endip 1.1.1.101

  next

  end

  FPX31 # show firewall policy

  config firewall policy

  edit 1

  set type explicit-web

  set uuid d36b62ec-c658-51ec-d4b8-df519a17e7f2

  set dstintf "any"

  set srcaddr "all"

  set dstaddr "all"

  set action accept

  set schedule "always"

  set service "webproxy"

  set explicit-web-proxy "web-proxy"

  set utm-status enable

  set logtraffic all

  set log-http-transaction enable

  set webproxy-forward-server "fw1"

  set poolname "pool1"

  set ssl-ssh-profile "deep-inspection"

  set av-profile "av1"

  next

  end

  FPX31 # dia sniffer packet any 'tcp[13] & 2 != 0'

  interfaces=[any]

  filters=[tcp[13] & 2 != 0]

  0.470567 10.10.1.2.54588 -> 10.10.1.31.8080: syn 2197052723

  0.470605 10.10.1.31.8080 -> 10.10.1.2.54588: syn 2228994176 ack 2197052724

  0.471099 1.1.1.100.4402 -> 10.10.1.32.8080: syn 2116725980

  0.471208 10.10.1.32.8080 -> 1.1.1.100.4402: syn 4133955111 ack 2116725981

  0.490317 10.10.1.21.54815 -> 10.10.1.31.8080: syn 2631998514

  0.490353 10.10.1.31.8080 -> 10.10.1.21.54815: syn 3576199856 ack 2631998515

  0.490946 1.1.1.101.1418 -> 10.10.1.32.8080: syn 832092590

  0.491086 10.10.1.32.8080 -> 1.1.1.101.1418: syn 343889646 ack 832092591

  0.510755 10.10.1.22.34259 -> 10.10.1.31.8080: syn 1288319710

  0.510792 10.10.1.31.8080 -> 10.10.1.22.34259: syn 1131257607 ack 1288319711

  0.511205 1.1.1.100.4406 -> 10.10.1.32.8080: syn 2875495461

  0.511339 10.10.1.32.8080 -> 1.1.1.100.4406: syn 378488109 ack 2875495462

  0.530717 10.10.1.23.40627 -> 10.10.1.31.8080: syn 2720076693

  0.530747 10.10.1.31.8080 -> 10.10.1.23.40627: syn 4127509262 ack 2720076694

  0.531136 1.1.1.101.1422 -> 10.10.1.32.8080: syn 2769098826

  0.531258 10.10.1.32.8080 -> 1.1.1.101.1422: syn 3900676280 ack 2769098827

  0.549988 10.10.1.24.53003 -> 10.10.1.31.8080: syn 1129467985

  0.550024 10.10.1.31.8080 -> 10.10.1.24.53003: syn 404621362 ack 1129467986

• Enable transparent mode:

  FPX31 # show firewall policy

  config firewall policy

  edit 1

  set type explicit-web

  set uuid d36b62ec-c658-51ec-d4b8-df519a17e7f2

  set dstintf "any"

  set srcaddr "all"

  set dstaddr "all"

  set action accept

  set schedule "always"

  set service "webproxy"

  set explicit-web-proxy "web-proxy"

  set transparent enable

  set utm-status enable

  set logtraffic all

  set log-http-transaction enable

  set webproxy-forward-server "fw1"

  set ssl-ssh-profile "deep-inspection"

  set av-profile "av1"

  next

  end

  FPX31 # dia sniffer packet any 'tcp[13] & 2 != 0'

  interfaces=[any]

  filters=[tcp[13] & 2 != 0]

  0.137139 10.10.1.23.50853 -> 10.10.1.31.8080: syn 2993591574

  0.137184 10.10.1.31.8080 -> 10.10.1.23.50853: syn 3859175252 ack 2993591575

  0.137537 10.10.1.23.3250 -> 10.10.1.32.8080: syn 1778004266

  0.137714 10.10.1.32.8080 -> 10.10.1.23.3250: syn 930837599 ack 1778004267

  0.157938 10.10.1.24.55807 -> 10.10.1.31.8080: syn 2979067078

  0.157980 10.10.1.31.8080 -> 10.10.1.24.55807: syn 986198629 ack 2979067079

  0.158421 10.10.1.24.24888 -> 10.10.1.32.8080: syn 2065406597

  0.158538 10.10.1.32.8080 -> 10.10.1.24.24888: syn 3326745164 ack 2065406598

  0.178692 10.10.1.2.55576 -> 10.10.1.31.8080: syn 4031359507

  0.178725 10.10.1.31.8080 -> 10.10.1.2.55576: syn 3038360258 ack 4031359508

  0.179179 10.10.1.2.10414 -> 10.10.1.32.8080: syn 356048076

  0.179296 10.10.1.32.8080 -> 10.10.1.2.10414: syn 3437380943 ack 356048077

  0.200084 10.10.1.21.33919 -> 10.10.1.31.8080: syn 1590490676

  0.200117 10.10.1.31.8080 -> 10.10.1.21.33919: syn 619246617 ack 1590490677

  0.200526 10.10.1.21.3254 -> 10.10.1.32.8080: syn 3543995028

  0.200707 10.10.1.32.8080 -> 10.10.1.21.3254: syn 922239342 ack 3543995029