URLScan 2.5 on ISA Server 2000 enhances the defensive capabilities of ISA Server 2000. Installing URLScan on ISA Server 2000 firewalls prevents suspicious HTTP requests from entering the internal network and mitigates threats to internal web servers. URLScan is highly configurable through the URLScan.ini file, which can be easily customized to meet a wide range of security and accessibility requirements. Consequently, when URLScan is installed on ISA Server, it plays a crucial role in a comprehensive defense-in-depth strategy that is designed to protect mission-critical applications such as SharePoint Portal Server 2003. This document explains how URLScan for ISA Server 2000 works and how to configure URLScan to protect your SharePoint Portal Server 2003 Web sites.

URLScan 2.5 is a powerful tool for protecting Web sites from invalid or deliberately malformed HTTP requests that can exploit potential weaknesses on Web sites. This document discusses how to install and configure URLScan 2.5 on ISA Server 2000 to protect SharePoint Portal Server 2003 and other web sites.

Because the configuration of URLScan will be, to some extent, unique to each environment, this document attempts to provide guidelines for implementing URLScan configurations, rather than providing a step by step set of configurations for given environments. The first part of this document provides an overview of URLScan and the attacks it prevents. The next part provides step-by-step instructions for installing and configuring URLScan on ISA Server. The last part provides information and guidance for troubleshooting and fine tuning URLScan configurations.

Overview of URLScan 2.5

URLScan 2.5 for ISA Server installs as a Web application filter that inspects HTTP traffic for invalid or suspicious requests and allows or denies these requests on the basis of configurable settings. The advantage of using URLScan 2.5 on ISA Server 2000 is that all HTTP traffic can be statefully inspected at the application layer and suspicious traffic blocked before it reaches the protected network.

As an example of suspicious traffic that URLScan 2.5 can protect against, imagine an attacker who creates and sends an HTTP request that appends a number of “..” characters in combination with “\” or “/” characters to a request for a virtual directory. Further imagine that the requested virtual directory is one likely to contain executable files or permit files to be executed. Such an exploit is illustrated by the following log entry of an attempted Nimda attack:

GET /scripts/..\xc1\x9c../winnt/system32/cmd.exe?/c+dir

This invalid request represents a directory traversal attack in which the attacker attempts to break out from the Web root directory into the file system and execute a command. When Web servers are protected against directory traversal attacks by disallowing HTTP requests that have too many dots or slashes, attackers may attempt to get around this protection by substituting Unicode characters for the slashes (this vulnerability is known as the Unicode translation vulnerability). Attackers can also send excessively long URLs in order to exploit potential vulnerabilities in unchecked buffers (this is a buffer overflow attack). Hackers may also attempt attacks that exploit functionality of HTTP 1.1 verbs (also known as HTTP methods or commands), such as PUT or DELETE, to alter files in virtual directories.

To defend against these and similar attacks, one should ensure that IIS has as small an attack surface as possible by applying the most recent security patches. You should also implement the suggestions and procedures outlined in the security configuration guidelines document, and/or upgrade to Windows 2003 and IIS 6.0, which provides significantly enhanced default security over previous versions of IIS.

You can significantly security for your published Web sites, such as installing URLScan 2.5 on an ISA Server 2000 firewall to prevent invalid or suspect URLs from being forwarded to internal Web servers in the first place. This is true even when those servers are using IIS 6.0 and have been hardened against HTTP-request exploits using the IIS version of URLScan and other methods. Please see the TechNet document on URLScan 2.5 for IIS for a comparison of the features of URLScan 2.5 and the built-in security features of IIS 6.0.

URLScan 2.5 for ISA Server 2000 is part of ISA Server 2000 Feature Pack 1, which is available as a free download. URLScan 2.5 installs as a high priority Web application filter on ISA Server. It is similar in functionality to URLScan 2.5 for Web servers, which is available as part of the IIS Lockdown tool or as a separate download.

However, some entries in the URLScan.ini file have no effect on URLScan processing on ISA Server 2000. For example, because URLScan 2.5 for ISA Server installs as a Web application filter that causes processing to occur immediately in the order specified in ISA Management console, enabling the Allowlatescanning option in the URLScan.ini has no effect. Also, because ISA Server 2000 Destination Sets used in combination with Web Publishing rules already mitigate directory traversal attacks, some options in the URLScan.ini will be superfluous. That said, these options should be configured as part of a comprehensive defense-in-depth strategy. Furthermore, URLScan 2.5 provides protection against directory traversal attacks for Web sites that are published through Server Publishing rules and lack this native level of protection afforded by Web Publishing rules.

URLScan 2.5 is configured through entries in a URLScan.ini file. Two different versions of the URLScan.ini are included in URLScan 2.5 for ISA Server 2000: URLScan_owa.ini and URLScan_iis.ini. These files are used as templates for the URLScan.ini file settings applied to publishing Outlook Web Access and Web sites respectively. The main difference between these two template files is their level of protection.

The settings in the URLScan_iis.ini file are more restrictive than the settings in the URLScan_owa.ini file. In particular, the URLScan_owa.ini allows a greater range of HTTP 1.1 verbs (methods) to support the WebDav methods required for Outlook Web Access. Using the URLScan_iis.ini file as the template for the URLScan.ini file causes Outlook Web Access to break. However, either file can be used as a template for the URLScan.ini file to secure a published SharePoint Web site. If you are publishing only a SharePoint Web site, you can use the more restrictive URLScan_iis.ini file as the template and relax or tighten its settings as necessary in your environment.

Note:
There can only be only one URLScan 2.5 configuration for all the published Web servers on the ISA Server. If different published Web sites have different security requirements, you will have to use a more general and relaxed URLScan configuration on the ISA Server and install more specific configurations of URLScan on the Web servers themselves.

You will likely need to customize the URLScan file, depending on your security requirements, to ensure both security and functionality of the published SharePoint Web site because there is no specific version of a URLScan.ini file for SharePoint portal services. However, the good news is SharePoint Portal Server 2003 runs on IIS 6.0 and is built on the .NET Framework. IIS 6.0 has much greater built-in security than previous versions of IIS. Consequently, IIS 6.0 provides a much smaller attack surface. This means a restrictive URLScan configuration is less likely to break a SharePoint Portal Server 2003 Web site, while still enabling users to perform sophisticated tasks.

Unlike the previous version of SharePoint Server, SharePoint Portal Server 2003 does not require WebDAV support on the Web server hosting the SharePoint site. One improvement of IIS 6.0 over IIS 5.0 is that you can “turn off” WebDAV, something not possible in IIS 5.0. Even without WebDAV installed on the Web server, users can connect to Web folders and manipulate files in these folders just as they could with the previous version of SharePoint server. Interestingly, this means users can still add, modify, and delete files in a SharePoint Web folder, even though URLScan may be blocking HTTP 1.1 and WebDAV verbs commonly used for these purposes, such as PUT or DELETE. Furthermore, an examination of the IIS log files reveal only a limited number of HTTP methods, primarily GET and POST, are used by external clients to perform tasks on SharePoint Server.

Tip:
To help determine optimal URLScan configurations, you should test external clients accessing the SharePoint site through ISA Server and then examine the IIS logs.

URLScan 2.5 Settings

The URLScan.ini file contains a number of sections where various settings are configured. These sections and the various setting appropriate to each are summarized below. It should be noted that a number of these settings have no effect when used in combination with the built-in security of IIS 6.0. That is, the built-in security of IIS 6.0 already protects against many of the same attacks as URLScan 2.5. However, the goal here is to prevent invalid HTTP requests from being forwarded to the internal network in the first place.

Note:
Improper settings in the URLScan.ini file will cause problems with published SharePoint and other Web sites. It is therefore important that you configure and verify appropriate URLScan.ini settings in a test lab before deploying URLScan on a production ISA Server.

The following is a detailed description and explanation of the URLscan.ini file settings.

[Options] Section

This [Options] section contains most of the primary settings for URLScan functionality. Entries in this section have the format OptionName=OptionValue. Below are the various URLScan options and their default values:

· AllowDotInPath=0
When this setting is 0, URLs containing more than one period (.) are not allowed. This protects against certain HTTP-request exploits that attempt to fool the Web server into processing a request for an otherwise blocked file extension. For example, the HTTP header contains a request for something like /NotAllowedExtension.asp/AllowedExtension.htm. This might be interpreted as a request for a file with an allowed .htm extension, which is passed to the PATH_INFO variable, when in fact it is a request for a blocked file extension (in this case, a .config extension). If URLScan is configured to block .asp files, but allow .htm files, this URL will be allowed if AllowDotInPath is set to 1. If this value is set to 0, directory names in the request path must not contain any periods, and the URL must not contain more than one period. It should be noted that URLScan always recognizes the first .exe, .dll, or .com extensions it sees in the request path.

· AllowHighBitCharacters=0
When this setting is 0, characters outside of the standard ASCII character set are not allowed. This mitigates weaknesses such as the Unicode translation vulnerability whereby hackers substitute Unicode characters for slashes in order to initiate a directory traversal attack. Disallowing high bit characters will limit request paths to those that use only ASCII characters and will consequently limit the number of allowable language-specific character sets.

· AlternateServerName=(not specified by default)
This setting is used as a masquerading/obscurity technique of somewhat limited value to impersonate another Web server type in the HTTP Server header responses. For example, the setting could be used to give the impression that the IIS Web server was an Apache Web server. The functionality of this setting depends on the setting for the RemoveServerHeader option.

· RemoveServerHeader=0
When this setting is 1, URLScan will remove the HTTP Server header from outgoing responses.

· EnableLogging=1
When this setting is 1, URLScan will log rejected URLs along with the reason for rejection in a file named URLScan.log that is created in the same directory containing the URLScan.dll file.

· LoggingDirectory=(not specified by default)
Specifies an alternate directory for the collection of log files. By default, URLScan log files are created in the same directory where the URLScan.dll is located.

· LogLongURLs=0
When this setting is 0, only the first 1024 bits of a URL are logged. When set to 1, a maximum of 128 kilobytes for each URL are logged.

· PerDayLogging=1
When set to 1, URLScan creates a new log file each day.

· PerProcessLogging=0
When set to 1, URLScan creates a separate log file for each process, appending the process ID to the log file name.

· NormalizeURLBeforeScan=1
A raw URL request may contain numbers preceded by a % sign to encode special characters, such as “%20” to indicate a space. When this setting is 1, which is the more secure setting, URLScan decodes (normalizes) the raw URL before processing it.

· VerifyNormalization=1
When this setting is 1, URLScan performs a second normalization on the decoded URL to mitigate double-encoding attacks that work by encoding the % sign.

· RejectResponseURL=(not specified by default)
Use of this setting allows the redirection of a rejected URL to an alternate location to create a customized message that is sent to the client providing the reason the URL was blocked. When no path is specified, the default location for the response is /<Rejected-By-URLScan>, which causes a 404 error to be sent to the client and which shows up on the IIS/ISA logs as failed request for /<Rejected-By-URLScan>. A key advantage of this setting is that it allows IIS or the Web Proxy Logs to record more detailed information. A useful feature of URLScan is that you can specify a path to an ASP file that generates the custom response, such as /path/customresponse.asp, even when URLScan is configure to otherwise block ASP files. When the path is set to a special value of /~*, URLScan will allow the requests to be processed, but will log them in the URLScan.log file. This is useful for testing purposes.

URLScan provides a number of server variables that can be used to create the custom response for the client:

o HTTP_URLSCAN_STATUS_HEADER. Provides the reason for rejection.

o HTTP_URLSCAN_ORIGINAL_VERB. Specifies the HTTP verb (PUT, POST, DELETE, GET, OPTIONS, CONNECT, etc.,) that caused the rejection

o HTTP_URLSCAN_ORIGINAL_URL. Specifies the rejected URL.

· UseAllowVerbs=1
When set to 1, URLScan will allow only URLs that contain verbs explicitly permitted in the [AllowVerbs] section of the file. When set to 0, URLScan will deny URLs on the basis of verb explicitly denied in the [DenyVerbs] section.

· UseAllowExtensions=0
Works like the UseAllowVerbs section. Used to allow or deny URLs on the basis of extensions of requested files listed in the [AllowExtensions] or [DenyExtensions] sections.

· UseFastPathReject=1
When set to 1, URLScan ignores the setting of RejectResponseURL and immediately sends an error message to the client browser without any further processing. The requested URL is still recorded in the Web Proxy log and the reason for the rejection in the URLScan.log on the ISA Server. The advantage of this setting is faster processing of rejected URLs.

[AllowVerbs] and [DenyVerbs] Sections

These sections are used to accept or deny HTTP traffic on the basis of verbs (methods) specified in the HTTP header. When UseAllowVerbs is set to 1, URLScan will accept traffic that contains only the verbs explicitly listed in the [AllowVerbs] sections. When UseAllowVerbs is set to 0, URL scan will reject traffic on the basis of the presence of verbs specified in [DenyVerbs] section.

Depending on the Web applications you wish to publish, the URLScan.ini file may have to allow both HTTP 1.1 and WebDAV verbs. HTTP 1.1 verbs include GET, POST, PUT, HEAD, DELETE, and OPTIONS. WebDAV verbs are an extension of HTTP 1.1 verbs. Some of these verbs are COPY, LOCK, MOVE, PROPFIND, PROPPATCH, SEARCH, and others. Consequently, administrators need to take care when configuring settings in either the [AllowVerbs] or [DenyVerbs] section to ensure URLScan is not configured so restrictively that it interferes with the functionality of the Web application.

[AllowExtensions] and [DenyExtensions] Sections

These sections are used to accept or deny HTTP requests on the basis of requests for files that have particular extensions. When UseAllowExtensions is set to 1, URLScan will accept requests for files that correspond to explicit settings in the [AllowExtensions] section.

SharePoint Portal Server 2003 is based on the .NET Framework. One consequence of this is that there is a greater range of file extensions that have to be taken into consideration when configuring URLScan. For example, an [AllowExtensions] section, if used, should include .aspx and .ashx extensions. The [DenyExtensions] section, if used, should possibly include .asax, .ascx, .config, .dll, .resx, xsd, xsx, and other extensions. The Microsoft Knowledge Base article, HOW TO: Configure URLScan to Protect ASP.NET Web Applications, provides a good starting point for learning more about this topic.

[DenyHeaders] Section

This section is used to deny specific HTTP headers in the request. This section is used primarily to prevent HTTP headers that could potentially invoke WebDAV from being forwarded to the Web server. For example, the URLScan_iis.ini template prevents the Translate, If, and Lock-Token headers from being forwarded to the internal network. But, this section could be used to deny any header, including non-standard headers.

[DenyURLSequence] Section

This section is used to deny URLs that contain specific strings. A primary use of this section is to prevent directory traversal attacks by specifying strings such as “..”, “\”, “/.”, “%”, and “&”. Note that entries in this section are processed after normalization so that URLs containing a % character will be allowed as long as they do not contain the % character after normalization.

[RequestLimits] Section

This section is used to set maximum lengths for specific parts of incoming requests. There are three settings that can be assigned values:

· MaxAllowedContentLength
Specifies the maximum value for the Content-Length. The default value is approximately 30 MBs, as reported by the URLScan initialization log file, and can be reduced significantly depending on the size of files that are uploaded to the Web server. It should be noted that a value specified here is not effective in limiting content sizes of a chunked-transfer encoded POST request. Chunked-transfer encoding can be disabled through URLScan by denying the Transfer-Encoding header in the [DenyHeaders] section.

· MaxURL
Specifies the maximum allowable length of the URL, excluding query strings that may be present. This value should be set to one representing the longest possible URL on the site.

· MaxQueryString.
Specifies the maximum allowable length of a query string in the URL. The default is 4 KB and should be reduced to reflect a more reasonable value.

A useful feature of this section is that it can also be used to restrict the length of any header by prepending the “Max-“ setting to the header name plus some value. For example, specifying “Max-Content-Type=100” and “Max-Referrer=100” will limit the length of both the Content-Type and Referrer headers to 100 characters. A value of 0 will cause URLScan not to enforce any limits on the header size. (To deny headers completely, use the [DenyHeaders] section.) Also, header sizes have to be explicitly listed here in order for URLScan to enforce size limits on them.

Step-by-Step: Installing Feature Pack 1 and URLScan 2.5

This section describes steps for installing Feature Pack 1 and then installing and configuring URLScan 2.5 on ISA Server 2000. You should note at the outset that these instructions only show how to set up a basic, working configuration of URLScan 2.5 for published SharePoint Portal Sites and do not necessarily represent a configuration providing the greatest functionality with the highest level of security.

Achieving the maximum level of security requires extensive prior experimentation in a test lab to determine optimal settings for a customized configuration that is predicated on the specific security requirements of your environment. In fact, because settings in URLScan can disable access to a published SharePoint Web site, you should test URLScan settings in a test lab before deploying it on a production ISA Server.

Step-by-Step Background Information

The test lab used to demonstrate these step-by-step instructions has the following configuration:

· Internal Network.
The internal network uses the 172.16.1.0/24 network ID. The default gateway for the network is 172.16.1.1, which is the internal IP address of the ISA Server.

· External Network
The external network uses a 192.168.100.0/24 network ID.

· Internal DNS and Active Directory Namespace
Internal.net is used as the Active Directory and DNS namespace for the internal network.

· Active Directory.
A Windows 2003 Active Directory domain controller named Ad1.internal.net is used to provide directory and DNS services. DNS is set up with root hints and forwarding to support resolution to the external network and the Internet. The IP address of the domain controller is 172.16.1.10.

· External DNS Namespace.
External.net is used as the DNS namespace for external clients connecting to resources published through ISA Server. The DNS zone files for external.net are located on the external network. A single host record points to external IP address of ISA Server to resolve the Fully Qualified Domain Name (FQDN) extranet.external.net for access to extranet SPS web site.

· SharePoint Portal Server 2003 Configuration.
SharePoint Portal Server 2003 installed with a co-located WMSDE database is set up on a Windows 2003 computer named Sps.internal.net.

· Virtual Web Site Configuration.
IIS 6.0 was installed on the Windows 2003 server as per the SharePoint Portal Server 2003 prerequisites found in the SharePoint Portal Server 2003 help files. An additional virtual Web site was set up on the SPS computer to support both intranet and extranet access to the same SPS configuration and content SQL databases. (Please see the SharePoint Portal Server 2003 help files for instructions on creating this configuration.) The intranet virtual web site is located at 172.16.1.11 and is configured to use Windows Integrated Authentication only. The extranet virtual web site is located at 172.16.1.12 and is configured to use Basic Authentication only. No DNS Resource Records resolve to the 172.16.1.12 address.

· ISA Server Configuration.
A stand-alone ISA Server 2000 with Service Pack 1 has been set up in integrated mode (Firewall and Web Proxy service) on a Windows 2000 member server with Service Pack 4. The basic configuration of ISA Server is as follows:

External NIC configuration:

IP address: 192.168.100.22/24

Default Gateway: 192.168.100.254/24

NetBIOS disabled

Registration of external IP address in Dynamic DNS zone disabled

DNS server: None

Internal NIC configuration:

IP address: 172.16.1.1/24

Default Gateway: None

NetBIOS enabled

Registration of IP address in Dynamic DNS zone enabled

DNS server: 172.16.1.10

ISA configuration details:

Local Address Table: 172.16.1.0 – 172.16.1.255

Packet Filtering: Enabled

Site and Content Rule: Default

Protocol Rule: Single rule added to provide access to all protocols

Destination Sets: Single Destination Set configured to support Web Publishing rule. Destination set points to extranet.external.net as FQDN.

Incoming Web Listener: Enabled with default settings. No authentication required.

Web Publishing Rule: Single Web publishing rule added to publish SPS extranet SharePoint virtual server (internal IP address = 172.16.1.12/24).

Client Configuration: All clients configured as S-NAT clients.

Installing and Configuring URLScan 2.5 for ISA Server 2000

URLScan 2.5 for ISA Server 2000 installs separately from Feature Pack 1. To install URLScan for ISA Server 2000, you must first install Service Pack 1 for ISA Server and then install Feature Pack 1. You can find both Service Pack 1 and Feature Pack 1 at the Microsoft ISA Server 2000 Web site. Feature Pack 1 is not available as a single download, and you must download multiple files to use all the features of Feature Pack 1. In particular, you must download URLScan 2.5 for ISA Server 2000 as a separate executable file.

Before installing Feature Pack 1, you should back up your ISA Server. Please consult the ISA Server and the Windows 2003 help files for procedures for backing up the ISA Server configuration and the Windows 2003 System State. Assuming you have installed Service Pack 1 for ISA Server and have downloaded and expanded the files for Feature Pack 1, the procedure for installing Feature Pack 1 and URLScan is as follows:

Click Start, then click Run, type cmd in the Run dialog box, and press Enter to open up a command prompt.
Type cd\<path-to-featurepack1-files-directory>, and press Enter.
To install Feature Pack 1, from the command prompt, type isafp1.exe, and press Enter. Feature Pack 1 prompts you to agree to the End User License Agreement (EULA) and then installs without requiring additional information.

To install URLScan 2.5, from the same command prompt, type isafp1ur.exe, and press Enter. You will be asked to confirm if you wish to continue with the installation of URLScan 2.5, as in Figure 1 below.

Figure 1 URLScan 2.5 Installation Alert

Click Yes on the Microsoft ISA Server 2000 Update Setup prompt. You will be presented with the option to use either the IIS or the Outlook Web Access URLScan template .ini file, as in Figure 2 below.

Figure 2 URLScan 2.5 Configuration File Selection

Click Yes on the Microsoft ISA Server 2000 Update Setup prompt to choose the OWA configuration. (If you are not publishing an OWA site, you can click No instead. This will result in a much more restrictive configuration, but external SharePoint users will still be able to perform many tasks.)

This completes the basic installation of Feature Pack 1 with URLScan 2.5. At this point, you will want to verify the installation and configuration of URLScan.