Configuring a Windows Server 2003-based ISA Server as a Secure Authenticating SMTP Relay

 

In the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring the Windows Server 2003-based ISA Server 2000 Firewall as a Filtering SMTP Relay you learned how to configure an SMTP relay on the ISA Server computer. That SMTP relay can accept incoming SMTP messages destined for your email domains and relay these messages to the Exchange Server on your internal network.

 

While that SMTP relay configuration works well when you want to allow Internet SMTP servers to forward mail to your Exchange Server, this configuration does not allow external users to send SMTP messages to any mail domain. The problem with allowing the SMTP relay to relay SMTP mail to all domains is that spammers can use the SMTP relay on the ISA Server firewall to forward spam to any domain on the Internet.

 

The solution to this problem is to create an authenticating SMTP relay that requires that users authenticate before the SMTP relay will relay mail. The authentication requirement prevents spammers from hijacking your SMTP server to forward spam email. Your users can use authenticate to the SMTP relay and send mail to your own email domains, or any other domain on the Internet.

 

Advantages of creating an authenticating SMTP relay include:

 

·         External users that do not log into a local ISP can use the authenticating SMTP relay to send SMTP messages

 

Many external users connect to the Internet via a wired or wireless link that does not require logging onto a local ISP. These links can be found in hotels, restaurants and airports. The service provider does not provide your users an SMTP server address. This can be a problem for users who use POP3/SMTP or IMAP4/SMTP clients. These users will be able to read their email but won’t be able to respond to it if they cannot access an SMTP server. Your authenticating SMTP server allows them to send and receive email.

 

·         The authenticating SMTP relay can be configured to force TLS security on SMTP connections

 

Most SMTP servers do not require any type of authentication. Almost all ISPs allow “on network” hosts send SMTP messages and do not allow SMTP relay for off network users. Your users will be on networks outside of your administrative control and you have no idea what level of security is applied to the network. Malicious types may be running network analyzers in an attempt to capture user passwords.

 

You can configure your authenticating SMTP server to require TLS encryption. This protects the user credentials and the data moving between the SMTP client and SMTP server. The secure connection prevents people listening to activity on the wire from stealing user passwords and content contained within email messages.

 

·         Even for users who log onto a local ISP that provides an SMTP server, you can force these users to use the authenticating SMTP relay so that sensitive corporate information is not passed through the Internet “in the clear

 

You may have users to connect to the Internet via a local ISP that provides them with an SMTP server address. You may wish to force your email clients to use your authenticating SMTP server to protect user credentials, and more importantly, the data contained in the SMTP messages. You can not be sure of the level of security of any network between the client and the authenticating SMTP server. The best course of action may be to require users to connect to your authenticating SMTP server and use TLS to protect the data.

 

You need to carry out the following procedures to creating your authenticating SMTP relay:

 

·         Install Windows Server 2003 on the machine that will be the ISA Server firewall/SMTP relay

·         Install a Certificate Server on your network or obtain a Web site certificate from a third party

·         Install the Internet Information Services (IIS) 6.0 SMTP services on the ISA Server firewall/SMTP relay computer

·         Disable Socket Pooling on the ISA Server firewall/SMTP relay

·         Create a second virtual SMTP server on the ISA Server firewall/SMTP relay

·         Request and install a Web site certificate that can be used to create and force the TLS connection between SMTP client and server and force TLS encryption

·         Configure the SMTP server to control relay and user authentication

·         Create Remote Domains for your own domains so that authenticated users can send mail to your internal domains and configure the Remote Domain to authenticate with the Exchange Server’s SMTP service

·         Install ISA Server 2000 on the Firewall/SMTP relay computer

·         Configure inbound and outbound packet filters to support the authenticating SMTP relay

·         Configure the SMTP client to use the authenticating SMTP relay and install the CA certificate into the client’s Trusted Root Certification Authorities certificate store

 

In this ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document we will assume that you want to provide unauthenticated and authenticated relay services. The unauthenticated SMTP relay server is used by Internet SMTP servers to send mail to mail domains under your administrative control, and the authenticated SMTP relay is used by your users who need to relay mail to domains that are under your administrative control and those that are not under your control.

 

However, we will review steps that both authenticating and non-authenticating SMTP relay have in common so that you can use this document to create an authenticating SMTP relay without first configuring a non-authenticating SMTP relay.

 

*       Note:
Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring the Windows Server 2003-based ISA Server 2000 Firewall as a Filtering SMTP Relay for detailed information on how to configure the non-authenticating SMTP relay.

 

The remainder of this ISA Server 2000 Exchange Server 2000/2003 Deployment Kit will cover the required procedures outlined in the list above.

 

 

 

Installing Windows Server 2003 on the Firewall Computer

 

The computer that will become the ISA Server 2000 firewall/SMTP relay must meet the following minimum requirements:

 

 

The ISA Server firewall and Web caching components work very well on very modest hardware. This is true even when the SMTP filter is enabled and protecting the published co-located SMTP server. However, the SMTP Message Screener can be very processor intensive. This is why I recommend that you use a processor with a minimum of rating of 1.5 MHz. This is especially true if you plan on running an authenticating and non-authenticating SMTP relay on the same computer.

 

Installing a Certificate Server on the Internal Network

 

The first thing you need to do before you install the certificate server is to install the IIS 6.0 Web server. The Web server component is required to host the Web enrollment site for the CA.

 

Installing the IIS 6.0 SMTP Services on the Windows Server 2003 Firewall Computer

 

The SMTP Message Screener requires the IIS SMTP service. You will need to install the SMTP service because Windows Server 2003 does not install IIS by default. Perform the following steps to install the IIS 6.0 SMTP service:

 

  1. Click Start, point to Control Panel and click the Add or Remove Programs command (figure 1).

 

Figure 1

 


  1. Click the Add/Remove Windows Components button on the left side of the Add or Remove Programs window (figure 2).

 

Figure 2

 

 


  1. In the Windows Components dialog box, click on the Application Server entry (do not put a checkmark in its checkbox!). Click on the Details button (figure 3).

 

Figure 3

 

 


  1. In the Application Server dialog box, click on the Internet Information Services entry (do not put a checkmark in its checkbox!). Click on the Details button (figure 4).

 

Figure 4

 

 


  1. On the Internet Information Services (IIS) page, put a checkmark in the SMTP Service checkbox. The Internet Information Services Manager checkbox will be automatically selected for you. Click OK.

 

Figure 5

 

 


  1. Click OK in the Application Server dialog box (figure 6).

 

Figure 6

 

 


  1. Click Next on the Windows Components page (figure 7).

 

Figure 7

 

 


  1. The Windows Components Wizard installs the IIS SMTP service (figure 8).

 

Figure 8

 

 

  1. Click Finish when the Wizard completes.

 

 

 

Disabling Socket Pooling on the ISA Server Firewall SMTP Relay Computer

 

 

You will need to disable socket pooling if you intend to use the Server Publishing method. Perform the following steps to disable socket pooling for the Windows Server 2003 IIS 6 SMTP service:

 

*       Note:
Socket pooling allows a service to listen on all IP addresses and all interfaces. This prevents Server Publishing Rules from binding to the socket required listen for incoming SMTP messages.

 

 

1.       Click Start and then click the Command Prompt link. In the Command Prompt window, switch to the Inetpub\AdminScripts folder. Then type in the following command and press ENTER (figure 9):

 

Adsutil.vbs set /smtpsvc/1/DisableSocketPooling 1

 

Figure 9

 

2.       If the SMTP service is installed and you entered the command correctly, you should see what appears in figure 10.

 

Figure 10

 

 

3.       Close the command prompt window.

 

The SMTP service will continue to listen on all IP addresses on all interfaces. You must configure the service to listen on specific IP addresses to limit the server to listening on a subset of addresses.

 

Creating a Second Virtual SMTP Server on the ISA Server 2000 Firewall SMTP Relay Computer

 

You must create a second SMTP virtual server for your authenticating SMTP relay. You cannot use a single virtual server for your authenticating and non-authenticating SMTP relay. There are two reasons for this:

 

 

You want to force TLS encryption for the SMTP traffic moving between the SMTP client and server. Internet SMTP servers sending mail to your domains will not negotiate TLS encryption with your SMTP relay. Because you want to force TLS encryption for all SMTP connections on the authenticating SMTP relay, you must create a second SMTP virtual server.

 

You also want to force authentication on the authenticating SMTP relay computer because this machine is capable of relaying mail to any email domain. Spammers will use your mail server to relay spam if you do not force authentication.

 

*       Note:
The second virtual SMTP server cannot listen on the same IP address as the first virtual SMTP server. If you choose to run both an authenticating and non-authenticating SMTP relay on the same computer, you must bind at least two IP addresses to the external interface of the ISA Server firewall/SMTP relay. If you choose to create only an authenticating SMTP relay, you do not need to create the second virtual SMTP server.

 

Perform the following steps to create the second virtual SMTP server on the ISA Server firewall/SMTP relay computer:

 

  1. Open the Internet Information Services (IIS) Manager console. Right click on your server name, point to New and click on SMTP Virtual Server (figure 11).

 

Figure 11

 


  1. Type in a name for your virtual server in the Name text box on the Welcome to the New SMTP Virtual Server Wizard page (figure 12).

 

Figure 12

 

 


  1. On the Select IP Address page, click the down arrow in the Select the IP address for this SMTP virtual server drop down list box (figure 13). Select an IP address on the external interface of the ISA Server firewall that is not being used by any other SMTP virtual server or any SMTP Server Publishing Rule.

 

*       Note:
No two virtual SMTP servers can listen on the same IP address. In addition, on virtual SMTP server can use an IP address that is already in use by an SMTP Server Publishing Rules. The SMTP Server Publishing Rule needs to bind the socket. If you configure the SMTP virtual server to use the same IP address as an SMTP Server Publishing Rule, the Publishing Rule will no longer function.

 

Figure 13

 


  1. On the Select Home Directory page, type in the path to a folder on the local computer that will store the SMTP server’s folder hierarchy. The Wizard will create the folder “on the fly”, so you do not need to create it in advance (figure 14). Click Next.

 

Figure 14

 

 


  1. On the Default Domain page, type in a bogus name in the Domain text box. We do not want this SMTP server to be the endpoint for any SMTP messages. This SMTP server’s only job is to relay messages. In this example, I’ve used the name SMTPAUTH to help me identify this virtual SMTP server. This name appears in SMTP headers sourcing from this virtual SMTP server and you can use this information when troubleshooting SMTP connections with Network Monitor (figure 15). Click Next.

 

Figure 15

 

 


  1. The new virtual SMTP server appears in the left pane of the Internet Information Services (IIS) console (figure 16).

 

Figure 16

 

 

Request and Install a Web Site Certificate on the Authenticated SMTP Relay Server

 

The authenticating SMTP relay server requires a certificate to create the TLS connection between itself and the SMTP client. There are several ways you can obtain a Web site certificate for the virtual SMTP server. The most convenient method is to obtain a certificate from an online certificate authority. Two conditions must be met in order to obtain a certificate from an online certificate authority:

 

·         You have installed an enterprise CA

·         The ISA Server firewall/SMTP relay belongs to the same domain as the enterprise CA

 

If the ISA Server firewall/SMTP relay does not belong to the same domain as the enterprise CA, then you must submit an offline request and manually request and install the Web site certificate.

 

*       Note:
For information on how to submit an offline request for a Web site certificate, please see ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document How to Obtain a Web Site Certificate

 

Perform the following steps to create the online request and install the certificate:

 

1.       In the Internet Information Service (IIS) Manager console, Right click on the authenticating virtual server’s name and click the Properties command (figure 17).

 

Figure 17

 

 

2.       In the authenticating virtual server’s Properties dialog box, click on the Access tab. On the Access tab, click on the Certificate button in the Secure communication frame (figure 18).

 

Figure 18

 

 

3.       Read the information on the Welcome to the Web Server Certificate Wizard page and click Next (figure 19).

 

Figure 19

 

 

4.       On the Server Certificate page, select the option that fits your requirements (figure 20). You have the following options:

 

Create a new certificate

This allows you to request a new certificate for the SMTP virtual server. If you do not already have a certificate, then this is the option you should select.

Assign an existing certificate

If you already have a certificate for this virtual server, then you can bind the certificate to the SMTP virtual server using this option. The certificate must already be installing into the machine’s certificate store

Import a certificate from a Key Manager backup file

If you have a certificate from an IIS 4.0 site, you can import the certificate from a Key Manager backup file using this option

Import a certificate from a .pfx file

If you have a certificate that has been exported with its private key into a .pfx file from another site, you can import that certificate into the machine’s certificate store and assign it to the virtual SMTP server

Copy or Move a certificate from a remote server to this site

If you have another server with the same certificate, and you want to use that same certificate on this virtual SMTP server, then select this option. The server should be located somewhere on the internal network.

 

We do not have a certificate for this virtual SMTP server, so we must request a new certificate. Select the Create a new certificate option and click Next.

 

Figure 20

 

 

5.       Select the Send the request immediately to an online certificate authority option on the Delayed or Immediate Request page (figure 21). This allows the Wizard to automatically forward the request to the enterprise CA on the internal network. The Prepare the request now, but send it later option creates a text file that you can submit to any CA and obtain a certificate. You must then manually install the certificate after you receive it. Click Next.

 

Figure 21

 

 

6.       Type in a “friendly name” in the Name text box on the Name and Security Settings page (figure 22). This is a descriptive name only and does not effect the functionality of the certificate. Chose a bit length for the encryption key. The longer the bit length, the more processor intensive the encryption process will be. The default value of 1024 is reasonably secure. Click Next.

 

Figure 22

 

 

7.       Type an Organization and Organizational unit name in the text boxes provided on the Organizational Information page (figure 23). Click Next.

 

Figure 23

 

 

8.       The Your Site’s Common Name page is very important and the correct Common name must be entered into the text box (figure 24). The common name is the name the client application users to connect to the site. For example, if the common name on the certificate is smtpauth.internal.net, then the client must connect to the virtual SMTP server using this name.

 

In addition, this name must resolved to the IP address that is listening for the virtual SMTP server that uses this certificate. In our current example the authenticating virtual SMTP server is listening on 131.107.0.3. The fully qualified domain name smtpauth.internal.net must resolve to 131.107.0.3 so that the client can send the request to the correct IP address the virtual SMTP server is listening on.

 

Note that the SMTP email client software must be configured to use the FQDN of the SMTP relay and not the IP address. The client need to match the name on the certificate the SMTP relay presents to it with the name that its connecting to. You will see an error message on the SMTP email client if these names do not match.

 

Enter the correct FQDN in the Common name text box and click Next.

 

 

Figure 24

 

9.       Type in a State/province and City/locality on the Geographical Information page (figure 25). Use the drop down list box to select a Country/Region. Click Next.

 

Figure 25

 

 

10.   Your enterprise CA will appear in the Certificate authorities drop down list box on the Choose a Certificate Authority page (figure 26). If you have more than a single enterprise CA on the network, you can choose one of them from the list. In this example we have a single enterprise CA, so we will go with the default. Click Next.

 

Figure 26

 

 

11.   Review the information on the Certificate Request Submission page (figure 27). Confirm that the Common Name (listed as the Issued To entry on this page) matches the name users will use to access this virtual SMTP server. Click Next.

 

Figure 27

 

 

12.   Click Finish on the Completing the Web Server Certificate Wizard page (figure 28).

 

Figure 28

 

 

13.   The SMTP virtual server now has a certificate installed that it can use to create the TLS sessions between itself and the SMTP email client. The next step is to force a TLS session so that SMTP email clients can’t create a non-secured connection. Click the Communication button in the Secure communication frame (figure 30).

 

Figure 30

 

 

14.   On the Security dialog box (figure 31), put a checkmark in the Require secure channel checkbox. This forces the clients to use TLS encryption. All Windows clients now support 128-bit encryption, so you can select the Require 128-bit encryption checkbox. However, not select that checkbox if you know that you have SMTP email clients that do not support 128-bit encryption, or if you are not sure. Click OK.

 

Figure 31

 

 

15.   Click Apply and then click OK in the SMTP virtual server’s Properties dialog box.

 

 

Configure the SMTP Server to Control Relay and User Authentication

 

This authenticating SMTP relay server needs to be able to route SMTP email to any SMTP server on the Internet. This would be considered an “open relay” if the authentication requirement were not in place. We therefore need to examine the relay configuration of this authenticating SMTP relay and configure its user authentication support.

 

Perform the following steps to configure the Default Virtual SMTP Server:

 

1.       Click Start, point to Administrative Tools and click on Internet Information Services (IIS) Manager. In the Internet Information Services (IIS) Manager window, expand your server name and click on the Default SMTP Virtual Server entry in the left pane. Right click on authenticating virtual SMTP server and click on the Properties command (figure 32).

 

Figure 32

 

2.       In the authenticating SMTP server’s Properties dialog box, click on the General tab. Click the down arrow in the IP address drop down list box. Note the list of IP addresses included in the list. You should see entries for you external addresses, internal addresses, an (All Unassigned) (figure 33).

 

Select an external IP address because you will be using packet filters to allow inbound access to your authenticating SMTP relay.

 

Click Apply after selecting an IP address to bind to the authenticating SMTP relay server.

 

Figure 33

 

 

3.       Click on the Access tab. You have a number of options available on this tab. Click on the Relay button that’s located in the Relay Restrictions frame (figure 34).

 

Figure 34

 

 

4.       The default setting in the Relay Restrictions allows no machines to relay through this virtual SMTP relay except for authenticated users (figure 35). This is a global setting for the virtual SMTP server.

 

We want only authenticated users to have “open relay” access to this machine. Leave the checkmark in the Allow all computers which successfully authenticate to relay, regardless of the list above checkbox. Removing this option would prevent this virtual server from being able to relay to any mail domain except for those mail domains you create Remote Domain entries for.

 

Click OK.

 

Figure 35

 

 

5.       Click on the Authentication button in the Access control frame (figure 36).

 

Figure 36

 

6.       The default setting in the Authentication dialog box is to allow Anonymous access (figure 37). We do not want to allow anonymous access to this virtual SMTP relay server because we want it to be able to relay to any domain on the Internet.

 

Remove the checkmark from the Anonymous access checkbox and place a checkmark in the Basic Authentication checkbox. All operating systems support basic authentication.  You will see a dialog box warning your that credentials will be passed in the clear without data encryption. We will be using TLS encryption so basic authentication will not cause a security issue. Click Yes (figure XX)

 

In the Authentication dialog box, put a checkmark in the Requires TLS encryption checkbox. This ensures that no one will be able to authenticate without successfully negotiating a TLS secured connection. Enter a Default domain name if you want a domain automatically appended to the account name.

 

Click OK.

 

Figure 37

 

Figure 38

 

 

7.       Click on the Delivery tab. On this tab you can configure how long the SMTP relay will wait before retrying to send messages to Internet SMTP servers. This “queuing” of SMTP messages is helpful when the destination SMTP server is offline at the time the use sends the message that needs to be relayed.

 

If the SMTP relay cannot immediately deliver the messages, it will place them in a queue and attempt to redeliver the messages based on the intervals set on this tab.

 

Note that the SMTP relay will continue to resend the mail indefinitely. After the third retry, subsequent delivery attempts are done at an interval based on the Subsequent retry interval (minutes) entry. Even if the destination SMTP server is down for a day or more, the SMTP relay will queue mail for you and be ready to deliver it when the server comes back on line.

 

Figure 39

 

8.       Click on the Advanced button on the Delivery tab (figure 39[above]). On the Advanced Delivery dialog box, enter the FQDN or IP address of a smart host. A smart host is an SMTP server that can relay mail for your SMTP relay. The advantage of the smart host is that your SMTP relay computer does not need to resolve the MX domain name to an IP address. The smart host will do that for your SMTP relay. In most cases, your ISP’s SMTP server can be used as a smart host.

 

You not need to use a smart host. You can allow your SMTP relay to resolve MX domain names to IP addresses itself. The ISA Server firewall must be configured with a DNS server address that allows it to resolve both internal and external Internet DNS names.

 

You can enter either a FQDN or an IP address in the Smart host text box. If you enter an IP address, make sure to include square brackets around the address, as seen in figure 40. If you use a FQDN, make sure the ISA Server firewall/ SMTP relay is able to resolve the name of the SMTP server.

 

Click OK.

 

Figure 40

 

9.       Click Apply and then click OK in the authenticating virtual SMTP server Properties dialog box.

 

 

Create Remote Domains to Support Your Email Domains and Enable Relay for Those Domains

 

The authenticating SMTP relay is server is now configured to relay messages to all Internet mail domains. This includes your internal domains hosted on the Exchange Server on your internal network if you have created a split DNS and configured the ISA Server firewall/SMTP relay to leverage the split DNS and resolve your email domains to the IP address of the Exchange Server on the internal network.

 

If you’re not comfortable with creating a split DNS infrastructure, or if you don’t want to create a split DNS, then you can use a Remote Domain to route SMTP messages for your email domains to the Exchange server on the internal network.  A Remote Domain is an email domain hosted on an SMTP server that isn’t the local SMTP server. In our situation, this SMTP server is the Exchange Server’s SMTP service.

 

For example, if you are hosting the email domain internal.net, then you want all email messages destined for you users in the internal.net email domain to be relayed by the SMTP relay server to the Exchange Server’s SMTP service on the internal network.

 

*       Note:
The remote domains do not need to be the same as your internal network’s Active Directory domain or domains. The email domains accepted by the Exchange Server’s SMTP service can be configured in the Recipient Policy of the Exchange Server. For example, the Exchange Server may be a member of the internal.net domain, but it can be configured to receive email destined for users in the domain.com and domain.net domains.

 

You need to create a Remote Domain for each email domain you want your Exchange Server to receive email for. In the current example, we want to host mail for a single email domain, internal.net.

 

Perform the following steps to create a Remote Domain for the internal.net domain:

 

1.       Click Start, point to Administrative Tools, and click on Internet Information Services (IIS) Manager. In the Internet Information Services (IIS) Manager console, expand your server name and then expand the Default SMTP Virtual Server node. Click on the Domain node and then right click on it. Point to New and click on Domain (figure 41).

 

Figure 41

 

 

2.       On the Welcome to the New SMTP Domain Wizard page of the New SMTP Domain Wizard, select the Remote option (figure 42). Click Next.

 

Figure 42

 

 

3.       On the Domain Name page, type the name of your email domain in the Name text box. Click Next (figure 43).

 

Figure 43

 

 

4.       The new Remote Domain appears in the right pane of the console (figure 44). Right click on the Remote Domain and click on the Properties command.

 

Figure 44

 

 

5.       In the Remote Domain’s Properties dialog box, click on the General tab (figure 45). On the General tab, put a checkmark in the Allow incoming mail to be relayed to this domain checkbox. This option allows mail addressed to users in this domain to be relayed to the Exchange Server’s SMTP service.

 

You have two options in the Route domain frame:

 

Use DNS to route to this domain This option allows your DNS infrastructure to route requests to your mail domains based on the MX record entries for these domains. In order for this to work correctly, you must have a split DNS infrastructure so that the ISA firewall machine can resolve the names of your email domains to the internal IP address of the Exchange Server computer. If the ISA Server firewall resolves the email domains to the external address of the ISA Server firewall, then the relay will fail.

 

*       Note:
Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Configuring DNS to Support Exchange Server Publishing for more information on how to configure your DNS infrastructure to support SMTP server publishing using ISA Server firewalls

 

Forward all mail to smart host This option allows you to enter the IP address of your Exchange Server and have mail for your domains relayed to this IP address. You must put brackets around the IP address. If you do not put brackets around the IP address, the SMTP relay server will attempt to resolve the IP address to an IP address.

 

The Outbound Security button allows you to configure authentication methods the SMTP relay server can use to authenticate with the SMTP service on the Exchange Server. In this example we will not configure the Remote Domain to authenticate with the Exchange Server because only mail destined for the domains under your administrative control are relayed to the server.

 

Click Apply and then click OK.

 

Figure 45

 

 

6.       You can force the SMTP virtual server to authenticate with the Exchange Server’s SMTP service by configuring authentication on the Outbound Security dialog box (figure 46). The default setting is Anonymous access. Select Integrated Windows Authentication and click the Browse button to find a user account that the SMTP virtual server can use to authenticate against the Exchange Server’s SMTP service. You can create a custom user account in the Active Directory for the SMTP virtual server.

 

After selecting the user account, the name of the account will appear in the Account text box. A line of asterisks will appear in the Password text box. However, these asterisk are just filler. The password is not automatically entered for you. You must type in the password for the account you selected.

 

Click OK.

 

Figure 46

 

7.       Click Apply and then click OK in the Remote Domain’s Properties dialog box (figure 47).

 

Figure 47

 

8.       In the Internet Information Services (IIS) Manager, right click on the Default SMTP Virtual Server node and click the Stop command (figure 48).

 

Figure 48

 

 

9.       In the Internet Information Services (IIS) Manager console, right click on the Default SMTP Virtual Server node and click the Start command (figure 49).

 

Figure 49

 

 

The SMTP relay is now ready to relay mail to your mail domain. If you have multiple email domains, you will need to create a Remote Domain for each of the email domains.

 

 

Installing ISA Server 2000 with the SMTP Filter and Message Screener on the Firewall Computer

 

The next step after installing and configuring the SMTP service on the ISA Server firewall is to install ISA Server 2000 with the SMTP Filter and Message Screener on to the Windows Server 2003 computer.

 

The ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Installing ISA Server 2000 on Windows Server 2003 for instructions on how to install ISA Server 2000 on Windows Server 2003. This document provides instructions on how to install all ISA Server 2000 components onto the Server. If you need to remove components, you can remove them later.

 

The ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document Installing and Configuring the ISA Server SMTP Filter and Message Screener provides details on how to enable and configure the SMTP filter and Message Screener.

 

 

 

Configuring Packet Filters or Server Publishing Rules on the ISA Server Firewall

 

We use packet filters to make the SMTP relay available to external users. The reason for this is that we want to enable the SMTP filter and the SMTP filter does not support TLS. We can bypass the SMTP filter by using a packet filter instead of a Server Publishing Rule.

 

The non-authenticating SMTP relay only requires a packet filter to allow inbound access to the SMTP relay. The reason is that the non-authenticating SMTP relay accepts SMTP mail for domains that you host on the internal network’s Exchange Server and forwards those packets to the Exchange Server. The non-authenticating SMTP relay does not send any packets outbound to Internet SMTP servers.

 

The authenticating SMTP relay requires a packet filter to allow inbound access to SMTP messages for the same reason the non-authenticating SMTP relay requires it. However, the authenticating SMTP relay requires a packet filter to allow it outbound access to SMTP servers on the Internet so that it can relay SMTP messages sent to it by your authenticated users.

 

You need to create two packet filters:

 

 

Configuring SMTP Packet Filters

 

Static packet filters allow external hosts to send packets to the external IP address on the ISA Server firewall’s TCP port 25 from any source port. This allows both SMTP servers and clients on the Internet to send SMTP messages to the SMTP relay on the ISA Server firewall.

 

Perform the following steps to create the inbound SMTP packet filter:

 

1.       Open the ISA Management console. Expand the Servers and Arrays node, then expand your server name. Expand the Access Policy node, click on the IP Packet Filters node and then right click on it. Point to New and click Filter.

 

Figure 50

 

2.       Type a name for the packet filter in the IP packet filter name text box on the Welcome to the New IP Packet Filter Wizard page (figure 51). Click Next.

 

Figure 51

 

 

3.       Select the Allow packet transmission option on the Filter Mode page (figure 52). Click Next.

 

Figure 52

 

 

4.       On the Filter Type page, select the Predefined option. Click the drop down list box and select the SMTP option (figure 53). Click Next.

 

Figure 53

 

 

5.       On the Local Computer page, select the Default IP addresses for each external interface on the ISA Server computer option. Click Next (figure 54).

 

Figure 54

 

 

6.       On the Remote Computers page, select the All remote computers option (figure 55). Click Next.

 

Figure 55

 

 

7.       Review the settings on the Completing the New IP Packet Filter Wizard page and click Finish.

 

Figure 56

 

 

8.       Double click on the packet filter you created to open its Properties dialog box. Click on the Filter Type tab. Notice that the Predefined option is automatically selected. The ISA Server firewall includes a number of preconfigured packet filters and the SMTP inbound packet filter is one of them. The important properties of this packet filter include:

 

Direction: Inbound

Local port: Fixed Port

Local port number: 25

Remote port: All ports

 

This packet filter allows incoming packets to TCP port 25. ISA Server firewalls use a dynamic packet filtering mechanism, so you do not need to create a second packet filter to allow the firewall to respond. Click OK to close the packet filter’s Properties dialog box.

 

Figure 57

 

The next step is to create the SMTP packet filter to allow outbound access to TCP port 25. Perform the following steps to allow outbound access to TCP port 25 so that your authenticated users can relay through the server:

 

 

1.       Open the ISA Management console. Expand the Servers and Arrays node, then expand your server name. Expand the Access Policy node, click on the IP Packet Filters node and then right click on it. Point to New and click Filter.

 

Figure 58

 

2.       Type a name for the packet filter in the IP packet filter name text box on the Welcome to the New IP Packet Filter Wizard page (figure 59). Click Next.

 

Figure 59

 

 

3.       Select the Allow packet transmission option on the Filter Mode page (figure 60). Click Next.

 

Figure 60

 

 

4.       On the Filter Type page, select the Custom option. Click Next.

 

Figure 61

 

 

5.       On the Filter Settings page, configure the packet filters with the following parameters:

 

IP protocol: TCP

Direction: Outbound

Local port: All ports

Remote port: Fixed port

Port number: 25

 

Click Next.

 

Figure 62

 

 

6.       Select the Default IP addresses for each external interface on the ISA Server computer option and click Next (figure 63).

 

*       Note:
The outbound SMTP filter must use the primary address bound to the external interface of the ISA Server firewall. This is true even when the authenticating SMTP relay is listening on one of the secondary addresses bound to the external interface of the ISA Server firewall/SMTP relay.

 

Figure 63

 

 

7.       Select the All remote computers option on the Remote Computers page and click Next (figure 64).

 

Figure 64

 

 

8.       Review the settings on the Completing the New IP Packet Filter Wizard page and click Finish (figure 65).

 

Figure 65

 

 

The packet filter will take effect immediately. You do not need to restart any of the ISA Server services or the ISA Server firewall computer.

 

Configuring the SMTP Client to use TLS Encryption for SMTP Messages

 

The SMTP client must be configured to negotiate a TLS connection with the authenticating SMTP relay. The method used to configure the client to use secure SMTP connections varies with the client. The following ISA Server 2000 Exchange Server 2000/2003 Deployment Kit documents discuss how to configure some popular SMTP client to the SMTP relay using TLS:

 

 

Regardless of the SMTP email client application, all clients will need a copy of the Root CA certificate of the CA that assigned the authenticating SMTP server its Web site certificate. Please refer to ISA Server 2000 Exchange Server 2000/2003 Deployment Kit document How to Import the Root CA Certificate into Email Client Certificate Stores.