ISA Server 2000 is an advanced application aware firewall. As a sophisticated application aware firewall, ISA Server 2000 examines application layer content of communications moving through it. You can use ISA Server 2000’s advanced application layer filtering to help prevent unwanted email, worms and viruses from endangering your network. In this ISA Server 2000 Application Layer Filtering Kit document you will learn about the how unwanted email, viruses and worms enter the network and how they damage computer networks and the businesses owning them. You will also learn how ISA Server 2000 application layer aware firewalls protects your network from these dangers. ISA Server 2000 application layer filtering firewalls can be placed virtually anywhere on a corporate network and this allows a high level of flexibility for organizations that already have an existing firewall infrastructure that they do not wish to replace.

You will learn where you can place the ISA Server 2000 firewall on your network, and see step by step details how to configure the ISA Server 2000 firewall as a secure unwanted email and virus/attachment filtering gateway.

The Problems: unwanted email, Virus and Buffer Overflow Attacks

Networks are under constant attack. There are almost as many different types of attacks as there are attackers. However, the three most common attacks made against networks and network services revolve around three main areas:

unwanted email
Viruses and Worms
Buffer Overflows

The a large proportion of problems encountered on networks today can be traced to one of these issues. unwanted email, viruses, worms and buffer overflows clog mail servers, disable network services, destroy data, consume available network bandwidth, and cost companies thousands and potentially millions of dollars.

unwanted email

Unsolicited commercial email (commonly known as “unwanted email”) is the greatest problem facing corporate networks and the Internet today. unwanted email leads to the following problems:

Wasted bandwidth on Internet connections
Increased Internet bandwidth cost
Increased non-productive traffic on the corporate network
Decreased employee productivity due to reading and deleting unwanted email
Increased administrative costs as network administrators attempt to reduce the negative effects of unwanted email
Increased disk usage on mail servers
Increased processor and memory utilization on mail servers
Increased exposure to legal liability secondary to users who may view offensive unwanted email messages
Increased risk of corporate servers and desktops being used by unwanted email relay stations

It has been estimated that unwanted email e-mail messages consume up to 50% of total bandwidth usage on the Internet today. Recent trends suggest there will be an acceleration in the unwanted email volume curve and an increase in the resources required to review, store, report and delete unwanted email messages from mail servers and user workstations. Corporate networks have already reached a breaking point regarding unwanted email on their mail servers. unwanted email control and elimination is no longer an optional network activity; it’s a requirement.

The challenge is to determine which e-mail messages are unwanted email. You want to block unwanted email and allow valid email messages into your mail systems. An overly aggressive approach to unwanted email control could have an adverse effect on overall productivity. There are a number of methods available used to control unwanted email and each of these methods inspect application layer information. Devices designed to control the influx of unwanted email must be application layer aware. Application layer aware devices can inspect the SMTP messages transporting the unwanted email email and evaluate characteristics of the messages. These characteristics include the following:

Source email address
Source email domain
Keywords in the subject line
Keywords in the message body
Attachment name
Attachment extension
Attachment size

unwanted email can be blocked by restricting access to your mail servers from domains known for hosting spammers. In some circumstances you may want to restrict a specific e-mail address rather than an entire domain. Blocking unwanted email based on source mail domain or e-mail address is only a first step in controlling unwanted email.

Another powerful method used to block unwanted email is keyword matching. unwanted email is blocked based on content in the subject line and message body. unwanted email messages typically contain words never or seldom used in legitimate e-mail. You can leverage this fact by blocking email messages containing targeted keywords in the subject or message body.

E-mail attachments can be included with both unwanted email and non-unwanted email email. Attachments present a special problem for corporate networks and are discussed in the next section on viruses and worms.

Viruses and Worms

Viruses and worms cause a tremendous amount of damage to corporate networks today. Viruses and worm attacks are responsible for:

Destruction of data on servers and workstations
Denial of service attacks on servers and workstations
Lost employee productivity because a workstation or network server is unavailable
Distribution of corporate secrets via mass mailing worms
Increased administrative costs due to repairing damaged workstations and servers
Increased bandwidth use on the corporate network and Internet connection secondary to mass mailing worms and denial of service attacks
Destruction of corporate Web sites
Lost sales because of service unavailability

Historically, worms and viruses were introduced into the corporate network by employees. Floppy disks and CDs containing infected files allowed exploits into the corporate network. Floppy disks and CDs now represent a minor source of infection. Internet downloads using a variety of protocols are now responsible for the vast majority of corporate virus and worm infestations. The most common Internet protocol used to download viruses and worms into the network is SMTP, which is used to send mail to the network.

Virus writers realize that email is a vital function to all businesses and they take advantage of this fact by crafting viruses and worms that spread via email. Both sophisticated and unsophisticated users open email attachments that contain dangerous code. The code is released to the user’s computer and then spreads to the rest of the network. A single infected host can damage virtually every networked device in a short period of time.

An effective way to prevent e-mail borne attacks is to block all attachments at the perimeter. An application aware device can then examine the attachments and perform one of the following actions:

Scan the attachment for viruses, worms and other dangerous code
Hold the attachment for later examination; this is sometimes referred to as “quarantine”
Forward the message, along with its attachment, to a security administrator’s email account for inspection and analysis
Delete the message immediately
Forward the message to the mail server if the application layer filter determines that the attachment does not represent a risk to network security

Note:
Application layer inspection of outbound mail is also possible. An organization may wish to block outgoing viruses and worms in an effort to protect other Internet connected networks. In addition, outbound mail inspection prevents users from sending attachment documents and other files that contain proprietary corporate data.

Buffer Overflow Attacks

Attackers use buffer overflow attacks to disable network servers. Unlike the typical virus or worm attack, a buffer overflow cannot be protected against by blocking attachments. Hackers use buffer overflow exploits to disable specific server services with the intent of creating a denial of service – either by disabling a specific service on the target computer or by taking the entire machine offline. More elaborate buffer overflow exploits can be used to disable key security features and allow the attacker to run commands of his choice on the targeted machine.

SearchSecurity.com defines a buffer overflow in this way:

“A buffer overflow occurs when a program or process tries to store more data in a buffer (temporary data storage area) than it was intended to hold. Since buffers are created to contain a finite amount of data, the extra information - which has to go somewhere - can overflow into adjacent buffers, corrupting or overwriting the valid data held in them. Although it may occur accidentally through programming error, buffer overflow is an increasingly common type of security attack on data integrity. In buffer overflow attacks, the extra data may contain codes designed to trigger specific actions, in effect sending new instructions to the attacked computer that could, for example, damage the user's files, change data, or disclose confidential information. Buffer overflow attacks are said to have arisen because the C programming language supplied the framework, and poor programming practices supplied the vulnerability.”

Buffer overflow attacks can be mounted against an organization’s SMTP server and can stop the inflow of mail. The best way to prevent a buffer overflow attack against the SMTP server is to stop the attacker at the network perimeter, before the exploit ever finds its way into the corporate network. An application aware device can evaluate the SMTP commands sent through the firewall and stop the attack.

The Solutions: The SMTP Filter and SMTP Message Screener

ISA Server 2000 is a sophisticated application layer aware firewall that can solve these problems. The ISA Server 2000 firewall can be configured to stop buffer overflow attacks, dangerous viruses and worms, and unwanted email at the network perimeter. ISA Server 2000 performs deep inspection of SMTP messages moving through the firewall and blocks dangerous code and unwanted email from entering the corporate network.

ISA Server 2000 firewalls use two technologies to protect the corporate network:

The SMTP filter
The SMTP Message Screener

When used in combination, the SMTP filter and SMTP Message Screener become powerful allies in the war against SMTP attacks and unwanted email.

The SMTP Filter

The ISA Server 2000 SMTP filter examines SMTP commands sent by Internet SMTP servers and clients. This application layer filter intercepts the SMTP commands and checks to see if they are larger than they should be. SMTP commands that are larger than RFC limits are assumed to be attacks against the SMTP server and are stopped at the perimeter by the ISA Server 2000 firewall’s SMTP application layer filter.

Figure A shows the flow of information and where the SMTP filter blocks the buffer overflow attack.

Figure B shows a list of the default commands included with the SMTP filer.

Each SMTP command has a Maximum Length associated with it. This length represents the number of bytes allowed for each command. If an attacker sends a command that exceeds the number of bytes allowed for the command, then the ISA Server 2000 firewall drops the connection and prevents the attacker from communicating with the corporate mail server.

The SMTP Message Screener

The SMTP Message Screener works together with the ISA Server 2000 SMTP filter, which is an application filter to intercept all SMTP traffic arriving on TCP port 25 of the ISA Server 2000 firewall. The filter accepts the traffic, performs deep application layer inspection, and passes it on to the corporate SMTP server only if it passes inspection based on rules you configure.

The SMTP Message Screener component can filter incoming mail based on:

Source e-mail address
Source e-mail domain
Keywords in the e-mail subject line
Keywords in the e-mail message body
Attachment name
Attachment file extension
Attachment size

An alert can be generated if mail is received from specific users. In addition, the SMTP Message Screener can be configured to hold the e-mail for later inspection or forward the message to a security administrator’s account for further examination and analysis.

Figure C shows an example of how the SMTP filter can be configured to block keywords. One of the most common unwanted email messages received on corporate networks contains the keyword Viagra. This example shows how to block e-mail containing the word Viagra in the message header (subject line) or body. There are three actions that can be taken when the e-mail matches this rule:

Delete message
Hold message
Forward message to

Placing the ISA Server 2000 Firewall on Your Network

The ISA Server 2000 firewall can be the only firewall on your network, or you can integrate ISA Server 2000’s powerful application layer filtering protection with your existing firewall infrastructure. Some of the common ISA Server 2000 network topologies are:

ISA Server 2000 acting as front-end firewall
ISA Server 2000 acting as back-end firewall
ISA Server 2000 acting as front-end and back-end firewalls
ISA Server 2000 acting as application layer filtering gateway in a perimeter network

ISA Server 2000 Front-end Firewall Topology

Smaller organization or organizations that do not already have a large investment in a current firewall infrastructure may prefer to make the ISA Server 2000 firewall a front-end firewall. The front-end firewall has a network interface on the corporate network and a network interface directly connected to the Internet. All communications into and out of the corporate network are exposed to ISA Server 2000’s deep application layer inspection.

The advantages of this configuration include:

All communications into and out of the corporate network are exposed to firewall policy
You only need to learn how to configure the ISA Server 2000 firewall software; this avoids the potential for firewall misconfiguration when multiple vendor firewalls are used
All inbound and outbound access can be controlled on a granular, user or group basis. Users only access the content and servers you want them to access, based on the rules you configure
This configuration is easy to set up and maintain

Figure D shows the network topology for the ISA Server 2000 front-end firewall placement.

ISA Server 2000 Back-end Firewall Topology

Organizations that already have an existing firewall infrastructure may prefer to leave the current firewalls in place and put the ISA Server 2000 firewall behind the current firewalls. This topology allows third party firewalls to provide high speed packet filtering (stateful filtering) before forwarding the remaining packets to the application aware ISA Server 2004 firewall. The network between the third party front-end firewalls and the ISA Server firewall is a perimeter network where publicly accessible services can be placed.

The third-party packet filtering firewalls have an interface directly connected to the Internet and an interface connected to a perimeter network between the third-party packet filtering firewalls and the ISA Server 2000 application layer aware firewall. The ISA Server 2000 firewall has an interface on the perimeter network and an interface on the protected, corporate LAN.

Advantages of this configuration include:

Organizations do not need to perform a major redesign of their current firewall infrastructures
Third party hardware-based firewalls can perform high-speed packet filtering. This offloads packet filtering overhead from the ISA Server 2000 firewall and increases the resources available on the ISA Server 2000 firewall to perform deep application layer inspection
Resources located on the corporate network are protected by the ISA Server 2000 firewall’s enhanced application layer inspection mechanisms
Granular inbound and outbound access control can be done on a user/group basis

Figure E shows the topology of the ISA Server 2000 back-end firewall topology.

ISA Server 2000 Front-end and Back-end Firewalls

The ISA Server 2000 front-end and back-end firewall configuration uses two ISA Server 2000 computers, one as the Internet edge firewall and the other as the corporate LAN edge firewall. The front-end ISA Server 2000 firewall has an interface directly connected to the Internet and an interface on the perimeter network between the firewalls. The back-end ISA Server 2000 firewall has an interface on the perimeter network and an interface on the protected, corporate LAN.

The advantages of this configuration include:

A single firewall system; this reduces the training overhead and the probability of a configuration error
Sophisticated application layer filtering protecting hosts on the perimeter network and the corporate network
You can leverage Web Proxy chaining and firewall chaining to significantly increase access control from perimeter network servers and users on the internal network. This prevents attackers from using compromised servers on the perimeter network as a launch point for outbound attacks from the perimeter network
Granular outbound user/group based access control for hosts on both the corporate network and the perimeter network
Excellent support for highly secure VPN passthrough, allowing access to protected resources on the corporate network

Figure F shows the topology for the ISA Server 2000 front-end back-end firewall configuration.

Figure F:

ISA Server 2000 Application Layer Filtering Web Proxy in the Perimeter Network

Some organizations already have an existing firewall infrastructure that includes front-end and back-end firewalls. These organizations have a large investment in their current firewall infrastructures and prefer to leave them intact. You can still leverage ISA Server 2000’s application layer filtering features by making the ISA Server an application layer filtering proxy. This ISA Server 2000 proxy can be placed on the perimeter network between the front-end and back-end third party packet filtering firewalls or you can place the ISA Server 2000 application layer proxy on the corporate network.

Advantages of the application layer filtering proxy configuration include:

The ability to leave the current firewall infrastructure intact; you can “drop in” the ISA Server 2000 application layer filtering proxy virtually anywhere
The third party front-end and back-end packet filtering firewalls can pass packets at high speed while allowing the ISA Server 2000 to provide a very high level of security for communications passed through its application layer inspection mechanisms
A hardened ISA Server 2000 proxy can be placed on the perimeter network segment to reduce the attack surface
In reverse Web Proxy scenarios, the ISA Server 2000 application layer filtering proxy can forward user credentials across the back-end firewall to pre-authenticate remote users

Figure G shows the topology of the application layer filtering proxy configuration.

SMTP Filter and Message Screener Scenarios: SMTP Relay and Message Screener on the Corporate Network

The SMTP filter always runs on the ISA Server 2000 firewall computer. However, you can place the SMTP Message Screener on another computer located on the protected network behind the ISA Server 2000 firewall. The SMTP Message Screener can be installed in the following locations:

On the ISA Server 2000 firewall itself
On an independent SMTP relay located behind the ISA Server 2000 firewall
On the Exchange Server

Message filtering requires a significant amount of processing power. For this reason, most organizations prefer to put the SMTP Message Screener on the ISA Server 2000 firewall computer or on an SMTP relay located somewhere on the corporate network, so as not to overload the Exchange server.

The SMTP Message Screener can be installed on an SMTP relay computer on the corporate network running the IIS 5.0 or IIS 6.0 SMTP service. The ISA Server 2000 firewall publishes the SMTP relay on the internal network and the Message Screener blocks dangerous e-mail at the SMTP relay computer. The SMTP Message Screener communicates with the SMTP filter to obtain information about which e-mail messages should be blocked.

Figure H shows the topology of the SMTP Message Screener on a dedicated SMTP relay configuration.

SMTP Filter and Message Screener Scenarios: SMTP Relay and Message Screener on the ISA Server 2000 Firewall

Many organizations prefer to use a “one box” solution in which the SMTP Message Screener is located on the ISA Server 2000 firewall itself. This simplifies setup and management of the SMTP Message Screener and reduces the hardware and software configuration overhead.

In this scenario, the ISA Server 2000 firewall acts as an SMTP relay. The IIS SMTP service is installed on the ISA Server 2000 firewall and processes the incoming SMTP Messages. The SMTP Message Screener filters unwanted email, viruses and attachments and relays the safe e-mail messages to the Exchange Server on the corporate network.

Note:
In both this scenario and the one where the SMTP Message Screener is installed on a dedicated SMTP relay, the ISA Server 2000 firewall can be integrated into an existing firewall infrastructure. The ISA Server 2000 firewall can act as a back-end firewall or an application layer filtering SMTP proxy located on the perimeter network. The only requirement is that the front-end firewall must forward inbound SMTP messages to the ISA Server 2000 firewall machine.

This is the most popular configuration because of the lower hardware and configuration overhead. The remainder of this document provides detailed step by step procedures for configuring the ISA Server 2000 firewall as a secure unwanted email and virus filtering SMTP relay.

Installing and Configuring the SMTP Message Screener and SMTP Filter on the ISA Server 2000 Firewall

The ISA Server 2000 firewall can be used as a unwanted email and virus filtering gateway. The following steps are required:

· Install the IIS 6.0 SMTP Service on the Windows Server 2003 ISA Server Firewall Computer

· Disable SMTP Service Socket Pooling

· Configure the IIS 6.0 SMTP Service Relay Properties

· Create Remote Domains to Support Your E-mail Domains and Enable Relay for Those Domains

· Install ISA Server 2000 onto the Windows Server 2003 Firewall Computer

· Configure Server Publishing Rules on the ISA Server Firewall

· Configure the SMTP Filter and SMTP Message Screener Properties

The remainder of this document provides detailed step by step procedures on how to configure the ISA Server 2000 firewall as a unwanted email and virus/attachment filtering gateway.

Install the IIS 6.0 SMTP Service on the Windows Server 2003 ISA Server Firewall Computer

The SMTP Message Screener requires the IIS SMTP service. You need to install the SMTP service because Windows Server 2003 does not install the SMTP service by default. Perform the following steps to install the IIS 6.0 SMTP service:

Click Start, point to Control Panel and click the Add or Remove Programs command (figure 1).

Figure 1

Click the Add/Remove Windows Components button on the left side of the Add or Remove Programs window (figure2).

Figure 2

In the Windows Components dialog box, click on the Application Server entry (do not put a checkmark in its checkbox!). Click on the Details button (figure 3).

Figure 3

In the Application Server dialog box, click on the Internet Information Services entry (do not put a checkmark in its checkbox). Click on the Details button (figure 4).

Figure 4

On the Internet Information Services (IIS) page, put a checkmark in the SMTP Service checkbox. The Internet Information Services Manager checkbox will be automatically selected for you. Click OK (figure 5).

Figure 5

Click OK in the Application Server dialog box (figure 6).

Figure 6

Click Next on the Windows Components page (figure 7).

Figure 7

The Windows Components Wizard installs the IIS SMTP service (figure 8).

Figure 8

Click Finish when the Wizard completes.

Disable SMTP Service Socket Pooling

Socket pooling is a feature designed to increase IIS performance, but when it is enabled, the SMTP service listens on all IP addresses on all adapters installed on the ISA Server firewall. Socket pooling is enabled by default. You must disable socket pooling to prevent the SMTP service from listening on all IP addresses on all adapters. Socket pooling prevents Server Publishing Rules from working correctly.

It is good practice to disable socket pooling for any IIS service installed on the ISA Server firewall. Perform the following steps to disable socket pooling for the IIS 6.0 SMTP service:

1. Click Start and then click the Command Prompt link. In the Command Prompt window, switch to the Inetpub\AdminScripts folder. Type the following command and press ENTER (figure 9):

Adsutil.vbs set /smtpsvc/1/DisableSocketPooling 1

Figure 9

2. If the SMTP service is installed and you entered the command correctly, you should see what appears in figure 10.

Figure 10

3. Close the command prompt window.

At this point the SMTP service continues to listen on all IP addresses on all interfaces. You must configure the service to listen on specific IP addresses to limit the server to listening on a subset of addresses. In the next section you will configure the SMTP service to listen on the internal IP address of the ISA Server 2000 firewall computer.

Configure the IIS 6.0 SMTP Service Relay Properties

The Default Virtual SMTP Server listens for incoming messages to e-mail domains you host. Perform the following steps to configure the Default Virtual SMTP Server:

1. Click Start, point to Administrative Tools and click on Internet Information Services (IIS) Manager. In the Internet Information Services (IIS) Manager window, expand your server name and click on the Default SMTP Virtual Server entry in the left pane. Right click on Default SMTP Virtual Server and click on the Properties command (figure 11).

Figure 11

2. In the Default SMTP Virtual Server Properties dialog box, click on the General tab. Click the down arrow in the IP address drop down list box. Note the list of IP addresses . You should see entries for your external addresses, internal addresses, and (All Unassigned) (figure 12).

Select an internal IP address. The Server Publishing Rule will forward the incoming SMTP packets to this address.

Click Apply after selecting an IP address.

Figure 12

3. Click the Access tab. There are a number of options available on this tab. Click on Relay button located in the Relay Restrictions frame (figure 13).

Figure 13

4. The default setting in the Relay Restrictions allows no relay through this virtual SMTP server except for authenticated users (figure 14). This is a global setting for the SMTP virtual server. We will override this global setting by configuring a Remote Domain on this SMTP virtual server later.

We do not want anyone to have “open relay” access to this machine, regardless of their ability to authenticate. Remove the checkmark from the Allow all computers which successfully authenticate to relay, regardless of the list above. Removing this option prevents this virtual server from being able to relay to any mail domains except for mail domains for which you create Remote Domain entries.

Click OK.

Figure 14

5. Click on the Messages tab. You have the option to limit the size of messages moving through the server, the number of messages per connection, and the number of recipients per message. You can also set a location for the badmail directory, which is the directory where messages not destined for any of your remote domains are deposited. Place this directory on a volume with a generous amount of free space so that your disk does not fill up in the event of a unwanted email flood.

Figure 15

6. Click on the Delivery tab (figure 16). On this tab you can configure how long the SMTP relay waits before retrying to send messages to your Exchange SMTP service. This allows “queuing” of SMTP messages on this SMTP virtual server when the Exchange Server is not available. If the SMTP relay cannot immediately deliver messages to your Exchange SMTP server, it will place them in a queue and attempt to redeliver the messages based on intervals set on this tab.

Note that the SMTP relay will continue to resend the mail indefinitely. After the third retry, subsequent delivery attempts are done at an interval based on the Subsequent retry interval (minutes) entry. Even if your Exchange Server is unavailable for a day or longer, the SMTP relay will queue mail for you. Once your Exchange Server becomes available, you can restart the SMTP service on the SMTP relay computer and the mail will be delivered to your Exchange Server’s SMTP service immediately.

Figure 16

7. Click on the Outbound Security button. In the Outbound Security dialog box (figure 17), you have the option to configure credentials the SMTP relay can use to authenticate with the SMTP service on the Exchange Server. This feature confers an additional level of security because then you can configure the SMTP service on the Exchange Server to block unauthenticated connection requests.

Click Cancel in the Outbound Security dialog box. We want to allow the SMTP relay to anonymously access the Exchange Server. You do not need to worry about spammers using your Exchange Server’s SMTP service as an “open relay”. The SMTP relay on the ISA Server 2000 firewall only relays messages that are destined for domains you host on your Exchange Server.

Figure 17

8. Click Apply and then click OK in the Default SMTP Virtual Server Properties dialog box.

Create Remote Domains to Support Your E-mail Domains and Enable Relay for Those Domains

The SMTP relay server is now configured to block all incoming SMTP messages. All incoming messages to the SMTP relay server are dropped. You do want to relay SMTP messages destined for domains hosted on the Exchange Server. This is accomplished by creating remote domains.

A Remote Domain is an e-mail domain hosted on the Exchange Server. For example, if you host the e-mail domain internal.net, then you want all e-mail messages destined for users in the internal.net email domain to be relayed by the SMTP relay server to the Exchange Server’s SMTP service on the internal network.

Note that e-mail domains do not need to be the same as your internal network’s Active Directory domain or domains. The e-mail domains hosted by the Exchange Server’s SMTP service can be configured in the Recipient Policy of the Exchange Server. For example, the Exchange Server might be a member of the internal.net domain, but it can be configured to receive e-mail destined for users in the domain.com and domain.net domains.

You need to create a Remote Domain for each e-mail domain for which you want your Exchange Server to receive e-mail. In the current example, we want to host mail for a single e-mail domain, internal.net.

Perform the following steps to create a Remote Domain for the internal.net domain:

1. Click Start, point to Administrative Tools, and click on Internet Information Services (IIS) Manager. In the Internet Information Services (IIS) Manager console, expand your server name and then expand the Default SMTP Virtual Server node. Click on the Domain node and then right click on it[DS1] . Point to New and click on Domain (figure 18).

Figure 18

2. On the Welcome to the New SMTP Domain Wizard page of the New SMTP Domain Wizard, select the Remote option (figure 19). Click Next.

Figure 19

3. On the Domain Name page, type the name of your e-mail domain in the Name text box. Click Next (figure 20).

Figure 20

4. The new Remote Domain appears in the right pane of the console (figure 21). Right click the Remote Domain and click on the Properties command.

Figure 21

5. In the Remote Domain’s Properties dialog box, click on the General tab (figure 22). On the General tab, put a checkmark in the Allow incoming mail to be relayed to this domain checkbox. This option allows mail addressed to users in this domain to be relayed to the Exchange Server’s SMTP service.

You have two options in the Route domain frame:

Use DNS to route to this domain This option allows your DNS infrastructure to route requests to your mail domains based on the MX record entries for these domains. In order for this to work correctly, you must have a split DNS infrastructure that allows the ISA firewall 2000 machine to resolve names of your e-mail domains to the internal IP address of the Exchange Server computer. If the ISA Server 2000 firewall resolves e-mail domains to the external address of the ISA Server 2000 firewall, the relay will fail.

Forward all mail to smart host This option allows you to enter the IP address of your Exchange Server and have mail for your domains relayed to this IP address. You must put brackets around the IP address. If you do not put brackets around the IP address, the SMTP relay server attempts to resolve the IP address to an IP address [sic].

The Outbound Security button allows you to configure authentication methods the SMTP relay server can use to authenticate with the SMTP service on the Exchange Server. In this example we will not configure the Remote Domain to authenticate with the Exchange Server because only mail destined for domains under your administrative control are relayed to the server.

Click Apply and then click OK.

Figure 22

6. In the Internet Information Services (IIS) Manager, right click on the Default SMTP Virtual Server node and click the Stop command (figure 23).

Figure 23

7. In the Internet Information Services (IIS) Manager console, right click on the Default SMTP Virtual Server node and click the Start command (figure 24).

Figure 24

The SMTP relay is now ready to relay mail to your mail domain. You will need to create a remote domain for each of your e-mail domains if you have multiple email domains.

Install ISA Server 2000 onto the Windows Server 2003 Firewall Computer

The next step after installing and configuring the SMTP service on the ISA Server firewall is to install ISA Server 2000 with the SMTP Filter and Message Screener onto the Windows Server 2003 computer. Please refer to the ISA Server 2000 installation instructions included on the CD and http://support.microsoft.com/default.aspx?scid=kb;en-us;331062 for information on running ISA Server 2000 on Windows Server 2003.

Configuring Server Publishing Rules on the ISA Server Firewall

You can use a Server Publishing Rule to make your SMTP relay available to external users. One of the main advantages of using a Server Publishing Rule is that it exposes the incoming connections to buffer overflow protection features included with the SMTP filter.

Perform the following steps to create the SMTP Server Publishing Rule:

1. Open the ISA Management console, expand the Servers and Arrays node and then expand the server node. Expand the Publishing node and click on the Server Publishing Rules node. Right click on the Server Publishing Rules node, point to New and click on Rule (figure 25).

Figure 25

2. Enter a name for the Server Publishing Rule in the Server publishing rule name text box on the Welcome to the New Server Publishing Rule Wizard page (figure 26). Click Next.

Figure 26

3. On the Address Mapping page, enter the IP address of the Exchange Server on the internal network in the IP address of internal server text box (figure 27). Click the Browse button to the right of the External IP address on ISA Server and select an address on the external interface of the ISA Server firewall that you want the accept the incoming SMTP messages. Select the IP address in the New Server Publishing Rule Wizard dialog box and then click OK.

Figure 27

4. Click Next on the Address Mapping page (figure 28).

Figure 28

5. On the Protocol Settings page, click the down arrow on the Apply the rule to this protocol drop down list box and select the SMTP Server entry (figure 29). Click Next.

Figure 29

6. On the Client Type page, select the Any request option (figure 30). Click Next.

Figure 30

7. Review your selections on the New Server Publishing Rule Wizard page and click Finish (figure 31).

Figure 31

8. The details of the new Server Publishing Rule appear in the right pane of the ISA Management console.

Figure 32

The ISA Server firewall and SMTP relay are now ready to accept incoming connections from external SMTP servers. All SMTP email messages destined for the remote domains you’ve configured on the SMTP relay will forward these messages to the Exchange Server on the internal network and the messages will appear in the users’ mailboxes.

Configure the SMTP Filter and SMTP Message Screener Properties

The SMTP filter and SMTP Message Screener configuration use the same interface, which can be found in the SMTP Filter Properties dialog box. However, the SMTP filter and SMTP Message Screener are two distinct entities. It is possible to use the SMTP filter and not use the SMTP Message Screener and it is possible to use the SMTP Message Screener and not use the SMTP filter.

For example, you can use the SMTP Filter without using the SMTP Message Screener by simply not installing the SMTP Message Screener. The SMTP filter then protects the published SMTP server against buffer overflow attacks, including the SMTP server co-located on the ISA Server firewall.

You can use the SMTP Message Screener and not the SMTP Filter by using an SMTP packet filter to allow inbound access to the SMTP relay. The SMTP Message Screener examines the incoming SMTP messages when they are accepted by the IIS SMTP service. The SMTP Filter will not be able to protect against buffer overflow attack because incoming SMTP messages accepted via a packet filter are not exposed to the SMTP filter.

Perform the following steps to configure the SMTP filter and SMTP Message Screener components:

1. Open the ISA Management console, expand the Servers and Arrays node and expand your server name. Expand the Extensions node and click on the Application Filters node. Right click on the SMTP Filter entry in the right pane of the console and click on the Properties command (figure 33).

Figure 33

2. The General tab is the first thing you see when the SMTP Filter Properties dialog box opens (figure 34). You can enable or disable the filter by adding or removing the checkmark in the Enable this filter checkbox. Click on the Keywords tab.

Figure 34

3. You can enter a prioritized list of keywords to filter on the Keywords tab. The SMTP Message Screener mediates the keyword filtering function. The SMTP filter does not examine SMTP messages for keywords. Click the Add button to add a keyword (figure 35).

Figure 35

4. Confirm that there is a checkmark in the Enable keyword rule checkbox (figure 36). Type in a keyword that you want the SMTP Message Screener to look for in the Keyword text box. Note the SMTP Message Screener does not search for whole words; it only looks at text strings.

Select one of the following options in the Apply action if keyword is found in frame:

Message header or body

If the keyword is found in either the message header or message body, then the Action you configure for the rule will be applied.

Message header

If the keyword is found in the header (subject line), then the Action you configure for the rule will be applied.

Message body

If the keyword is found in the body of the message, then the Action you configure for the rule will be applied

Click the down arrow for the Action drop down list box. You have the following options:

Delete message

The e-mail message is deleted without being saved or informing anyone that it has been deleted.

Hold Message

The SMTP message is held in the BADMAIL directory in the SMTP service’s folder hierarchy. You can view components of the held message, but the message is not saved in a format that you can easily forward to the recipient.

Forward message to

The SMTP message is forwarded to an email address you configure in this rule. Each rule can have a different email address to which the messages are forwarded. This allows you to forward the messages to an administrative account, from which they can easily be forward to the recipient if found to be acceptable.

Click OK on the Mail Keyword Rule dialog box after entering a keyword and action.

Figure 36

5. The keyword rule appears in the keywords list on the Keywords tab (figure 37). Click on the Users / Domains tab.

Figure 37

6. You can configure the SMTP Message Screener to block messages based on the sender’s user account or e-mail domain on the Users / Domains tab (figure 38). Enter a user e-mail account in the Sender’s name text box and click Add. The sender’s e-mail address appears in the Rejected Sender’s list. Enter an e-mail domain in the Domain name text box and click Add. The e-mail domain appears in the Rejected Domains list.

E-mail messages processed by the SMTP Message Screener matching e-mail addresses or e-mail domains found in these lists are deleted. These messages are not stored anywhere on the server, nor are they forwarded to any user or administrator. If a message from a rejected sender or rejected domain also contains a keyword matching a keyword rule, and that keyword rule is configured to hold the message, the message will not be held because it is rejected before the keyword search begins.

Click Apply and then click OK. Click on the Attachments tab.

Figure 38

7. You can block messages with certain types of attachments on the Attachments tab (figure 39). Click Add to add an attachment rule.

Figure 39

8. Confirm that there is a checkmark in the Enable attachment rule checkbox on the Mail Attachment Rule dialog box (figure 40). You have three options in the Apply action to messages containing attachments with one of these properties frame:

Attachment name

Select this option and enter a name for the attachment, including file name and file extension. Use this option when you want to block a specific file name but you don’t want to block all attachments with that particular file extension. For example, you do not want to block all .zip files, but you do want to block a file named exploit.zip.

Attachment extension

It is more common to block all files with a specific file extension. For example, if you want to block all attachments with the exe file extension, select this option and then type in either exe or .exe in the text box to the right of this option.

Attachment size limit (in bytes)

You can also block attachments based on their size. Select this option and type in the size of the file extension you want to block.

Click the down arrow for the Action drop down list box. You have the following options:

Delete message

The SMTP message is deleted without being saved or informing anyone that it has been deleted.

Hold Message

Forward message to

The SMTP message is forwarded to an e-mail address you configure in this rule. Each rule can have a different email address to which the messages are forwarded.

In this example we’ll select the Forward message to option so that you can see how to enter the forwarding address.

Figure 40

9. When you select the Forward message to option, a text box appears allowing you to enter an e-mail address to which you want to forward the message. However, the ISA Server must be able to resolve the address of the mail domain of this user.

For example, in figure 41 we have entered the e-mail address smtpsecurityadmin@internal.net. The ISA Server 2000 firewall must be able to access an MX record for the internal.net domain. The ISA Server firewall forwards the message to the SMTP server responsible for internal.net mail based on the information in the MX record.

In this example, the firewall is configured with an address of an internal network DNS server that can resolve both internal and external network names. The message is forwarded to the internal address of the Exchange server. You must configure a split DNS infrastructure if the internal.net domain is available to both internal and external users.

Note:
Please refer to the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring DNS to Support Exchange Server Publishing for information on how to create a split DNS to support SMTP server publishing.

Click OK in the Mail Attachment Rule dialog box. Click on the SMTP Commands tab.

Figure 41

10. The settings on the SMTP Commands tab are mediated by the SMTP filter component. The SMTP Message Screener does not evaluate SMTP commands and it does not protect against buffer overflow conditions. The commands in the list are limited to a pre-defined length. The connection is dropped if an incoming SMTP connection sends a command exceeding the allowed length. In addition, if a command not on this list is sent over the SMTP channel, it is dropped.

Click the Add button to add an SMTP command to the list (figure 42).

Figure 42

11. You might want to enter the AUTH command into the list of allowed SMTP commands. This is required if you want to allow external users to authenticate with an SMTP server published via an SMTP Server Publishing Rule. Users will not be able to authenticate with a SMTP server published via an SMTP Server Publishing Rule if the AUTH command is not added to the list and the SMTP filter is enabled.

Confirm that the Enable an SMTP command checkbox is checked. Type AUTH in the Command Name text box. Type 1024 in the Maximum Length Bytes text box. Click OK in the SMTP Command Rule dialog box (figure 43).

Figure 43

12. The new command appears in the list of SMTP commands on the SMTP Commands tab (figure 44). Click Apply and then click OK.

Figure 44

13. Close the ISA Server Management console.

14. Restart the ISA Server 2000 machine.

The ISA Server firewall/SMTP server is now ready to filter SMTP messages based on the parameters you set for the SMTP filter and SMTP Message Screener.

Summary

ISA Server 2000 is an advanced application aware firewall that examines the content of packets moving through it. You can use ISA Server 2000’s advanced application layer filtering to block unwanted email and viruses from endangering your network. In this ISA Server 2000 unwanted email and Attack Prevention Kit document, you learned about the problem of unwanted email and viruses, how to place the ISA Server 2000 firewall on your network, and how to configure the ISA Server 2000 firewall as a secure unwanted email and virus/attachment filtering gateway.