Creating a DNS Infrastructure to Support Exchange Server 2003

by [Published on 23 Oct. 2007 / Last Updated on 20 May 2013]

DNS troubleshooting in relation to configuring remote access to Microsoft Exchange Servers using ISA Server 2004.

Configuring remote access to Microsoft Exchange Servers using ISA Server 2004 is relatively easy because of the intuitive nature of ISA Server 2004 configuration interface. However, in spite of the ease of firewall configuration, one issue consistently creates problems for the ISA Server firewall administrator: DNS.

Discuss this article

DNS name resolution issues continue to be problematic for large numbers of network and firewall administrators because of the inherent complexity of DNS design and deployment. To help solve some of the more common DNS related problems, this article will cover the following issues:

Exchange Server 2003 outbound name resolution issues

The Exchange Server must be able to resolve Internet MX domain names so that it can forward SMTP messages to the correct mail server on the Internet. There are several ways this can be configured and we’ll discuss several of these options.

Remote access client name resolution issues

There are a number of common DNS name resolution issues that must be considered when remote clients connect to Exchange Server resources on the Internal network. In addition, you must also consider issues related to machines that move between the corporate network and remote locations.

Creating a DNS Infrastructure to Support the Exchange Server 2003 for Outbound Access

The Exchange Server’s SMTP service must be able to resolve the MX domain name for SMTP messages it relays. The Exchange Server must be able to relay mail for all domains for which the Exchange Server is not responsible. For example, if the Exchange Server is responsible for mail to users in the msfirewall.org domain, then all mail destined to mail domains other than msfirewall.org domain must be forwarded (relayed) to another SMTP server.

In order to relay mail, the Exchange Server must be able to determine the IP address of the mail server responsible for the destination domain’s e-mail. If the Exchange Server cannot determine the mail server’s address, the messages destined for that domain will remain in the message queue and then moved into the Badmail folder after a period of time.

The SMTP service uses information contained in a DNS MX record to deliver the message. A DNS MX record contains the name of a server; that server must also have a Host (A) record mapping the actual IP address used by that server. The server listed in the MX record for a mail domain is the server responsible for receiving mail for that particular domain.

The SMTP service must be able to find the name of the server contained in the MX record and then find the IP address of the name found in the MX record. This is done via DNS queries. The SMTP service can use one of several methods to resolve the MX domain name. These methods include:

  • Query an internal network DNS server that is configured on the Exchange Server’s network interface card’s TCP/IP settings
  • Query an external network DNS server (such as the ISP’s DNS server) that is configured on the Exchange Server’s network interface’s TCP/IP settings
  • Query an external DNS server that is configured in the Exchange Server’s SMTP service’s Properties dialog box
  • Use a smart host to completely offload the responsibility for name resolution to the smart host; a smart host is an upstream SMTP server that acts as an outbound SMTP relay

Query an Internal Network DNS Server to Resolve the MX Domain Name

By default, the Exchange Server’s SMTP service uses the DNS server configured on its network interface. Because the Exchange Server computer must belong to a Windows Active Directory domain, it must be configured to use an internal network DNS server to contact the Active Directory. The DNS server the Exchange Server uses to contact the Active Directory can be configured to resolve MX domain names for Internet mail domains.

The internal network DNS server must be set up to resolve Internet MX domain names either by performing recursion on its own, or by using a DNS forwarder. Allowing an internal network DNS server to perform recursion on its own requires the internal network DNS server be in direct contact with multiple Internet DNS servers. This situation increases the chance of the internal network DNS server of being compromised by a DNS exploit.

Instead of allowing the internal network DNS server to perform recursion, you can configure the DNS server to use a forwarder. In this case the DNS server forwards requests for all domains for which it is not authoritative. These name resolution requests are forwarded to another DNS server.

The ideal configuration is to set up the ISA Server firewall as a caching-only DNS server and configure the internal network DNS server to use the ISA Server firewall’s caching-only DNS server as its forwarder. This protects the DNS server on the internal network, and the zones contained within it, from attack from external DNS servers. It also allows you to focus your attention on hardening the caching-only DNS services located on the ISA Server firewall.

This configuration, where the internal network DNS server uses a caching-only DNS server on the ISA Server firewall as a forwarder is a preferred configuration for small to medium sized networks. DNS query traffic could become prohibitive on larger networks.

Query an External Network DNS Server to Resolve the MX Domain Name

It is possible to include the IP address of an external DNS server on the Exchange Server’s network interface’s DNS server. For example, some network administrators may configure the network interface on the Exchange Server computer with the IP address of a DNS server that allows the Exchange Server to contact the Active Directory and the IP address of a DNS server on an external network (such as the ISP’s DNS server).

I mention this option only to warn you against it. You should never bind an external DNS server address on the Exchange Server’s network interface. This type of configuration can lead to the Exchange Server being unable to contact the Active Directory and impairing the Exchange Server’s functionality.

Do not configure the Exchange Server’s network interface card with the IP address of an external DNS server. Instead, use the configuration described in the next section where you configure the SMTP service to use an external DNS server.

Query an Exchange SMTP Service External DNS Server

Because of the issues related to configuring the network interface card with the IP address of an external DNS server, you can configure the Exchange Server’s SMTP service to use DNS servers on an external network that are used only by the SMTP service and allows all other services on the Exchange Server computer to use an internal DNS server for all other name resolution services.

This configuration allows the Exchange Server’s network interface to be configured with the IP address of a DNS server on the internal network. This DNS server allows the Exchange Server to communicate with the Active Directory. However, this internal network DNS server is not able to resolve Internet host names. The Exchange Server’s SMTP service doesn’t use the internal DNS server to resolve Internet host names (in this case, Internet MX domain names). Instead, it uses its own custom configured DNS server.

You can find this custom SMTP service DNS server configuration by performing the following steps:

  1. Open the Exchange System Manager, expand the Servers node and then expand your server name.
  2. Expand the Protocols node and then expand the SMTP node. Right click on the Default SMTP Virtual Server and click the Properties command.
  3. Click on the Delivery tab. On the Delivery tab, click on the Advanced button.


Figure 1

  1. On the Advanced Delivery dialog box, click the Configure button that lies to the right of the Configure external DNS Servers description.


Figure 2

  1. In the Configure dialog box, click the Add button. In the Add dialog box, enter the IP address of an external DNS server and click OK.


Figure 3

  1. Repeat the procedure if you want to add more DNS external DNS servers. Click OK in the Configure dialog box.
  2. Click OK in the Advanced Delivery dialog box.
  3. Click Apply and then click OK in the Default SMTP Virtual Server Properties dialog box.
  4. Restart the SMTP service.

Now the SMTP service uses this external DNS server to resolve MX domain names. No other service on the Exchange Server computer will use this DNS server. All other services on the Exchange Server computer use the DNS server configured on the Exchange Server’s network interface card.

Use a Smart Host to Offload MX Domain Name Resolution

A smart host is an SMTP server that receives outbound SMTP messages from another SMTP server and forwards the SMTP messages to the correct destination after resolving the MX domain name. The smart host offloads the responsibility for resolving MX domain names from the original SMTP service to the smart host’s SMTP service.

For example, suppose our Exchange Server is responsible for mail in the msfirewall.org domain. The Exchange Server receives a message for a user in the domain.com domain. The SMTP service does not try to resolve the MX domain name for domain.com. Instead, the Exchange Server’s SMTP service forwards all mail not destined to the msfirewall.org domain to the smart host. The message for the user at domain.com is sent to the smart host. The smart host resolves the MX domain name for domain.com and forwards the message to the SMTP server responsible for the domain.com mail.

There are several advantages to using a smart host:

  • The Exchange Server does not need to generate traffic DNS name resolution traffic
  • You do not need to configure a DNS server on the internal network to resolve Internet host names
  • You do not need to configure the Exchange Server’s SMTP service to use a custom “external” DNS server

Discuss this article

The smart host can be another SMTP server on the internal network, or an SMTP server on another network, such as a perimeter network segment or your ISP. You can even use the IIS SMTP service on the ISA Server firewall as a smart host. If you have a reliable ISP that you trust to maintain high quality and high performance SMTP services, you might prefer to use their SMTP server as a smart host. You could also use an SMTP relay computer on your internal network as a smart host. The advantage of this approach is that you can configure the SMTP relay under your administrative control to filter outbound mail for key words and attachments using the ISA Server 2004 SMTP Message Screener.

You can configure the Exchange Server to use a smart host by performing the following steps:

  1. Open the Exchange System Manager, expand the Servers node and then expand your server name.
  2. Expand the Protocols node and then expand the SMTP node. Right click on the Default SMTP Virtual Server and click the Properties command.
  3. Click on the Delivery tab. On the Delivery tab, click on the Advanced button.


Figure 4

  1. On the Advanced Delivery dialog box, enter the IP address or the name of the smart host SMTP server. If you enter a name, make sure that the Exchange Server can resolve this name to the correct IP address. If you use an IP address, make sure you surround that IP address with straight brackets, as seen in the figure below.


Figure 5

Creating a DNS Infrastructure to Support Remote Access Clients

Remote clients need to able to resolve the name of the Exchange Server’s services to addresses that are accessible via the Internet. This can be problematic when the client machines are configured to use the private name of the Exchange Server, which usually is not resolvable to an Internet accessible IP address.

Roving users further complicate the problem. If the email client is located on the internal network, it needs to know the actual P address of the Exchange Server. When the email client is on an external network, the machine needs to know the IP address on the external interface of the ISA Server firewall used in the Web or Server Publishing Rule that publishes the Exchange services to the Internet.

We cover the following issues in this section on designing and configuring DNS to support remote email clients:

  • Split DNS infrastructure to support machines moving between the internal and external network
  • Special Considerations for secure SMTP, POP3, IMAP4, NNTP and SSL connections
  • Special Considerations for SMTP relay computers, including those on DMZ segments
  • Special considerations for Outlook RPC clients

Split DNS Infrastructure

A split DNS infrastructure allows email clients to move between the internal network and remote locations without requiring users to change the settings in their email client applications. The split DNS allows e-mail clients to properly resolve the name of Exchange Server regardless of location. The split DNS infrastructure allows users email experience to be location-independent and greatly enhances the user’s email experience.

Let’s look at some examples to see how a split DNS infrastructure is superior to the traditional approach of using different domain names for internally and externally accessible resources.

The first example is of an organization uses the name domain.local on the internal network and domain.net for resources accessible to external network hosts. Hosts on the internal network connect to the Exchange Server using the name mail.domain.local and remote users at external network locations use the name mail.domain.net.

You need two DNS zones to make this traditional, non-split DNS infrastructure work. One DNS zone contains the domain.local DNS zone and the other DNS zone contains the domain.net DNS zone. The DNS server with the domain.local DNS zone is used by internal network hosts and this DNS server is located somewhere on the internal network (behind the ISA Server firewall). The DNS zone containing the domain.net DNS zone is located either on an external network location, such as your ISP, or is located on the internal network and published via an ISA Server DNS Server Publishing Rule.

A single DNS server can be used to support the two DNS zones. The domain.local and the domain.net zones can be configured on the same Windows DNS server. As you’ll see later, a split DNS infrastructure requires that you have two physically separate DNS servers.

The internal and external DNS zones contain records that allow their respective clients access to the Exchange Server. The internal DNS zone containing the domain.local domain contains a Host (A) resource record mapping the name mail.domain.local to 10.0.0.2. The external DNS zone containing the domain.net zone maps the name mail.domain.net to the public IP address used in Server and Web Publishing Rules to allow inbound access to the Exchange Server, which in this case is 131.107.0.1.

This setup works fine for clients that always remain either on the internal network or at a remote location. Users who are always on the internal network configure their email applications to use mail.domain.local to access the Exchange Server’s services. Users who are always located on a remote network configure their email client applications to use the name mail.domain.net to access the Exchange Server’s services.

This setup fails when users move from an external network to the local network and when they move from the local network to an external network. For example, suppose your Senior VP of Sales travels to a customer site three thousand miles away. His laptop computer is configured to use the name mail.domain.local to connect to the Exchange Server’s services. What happens when the VP tries to access his email from the remote location?

The VP won’t be able to access his mail from the remote location because there is no way for his computer to resolve the name mail.domain.local. The .local domain is not valid on the Internet. To solve this problem, the VP must reconfigure his email client application and change all the server references from mail.domain.local to mail.domain.net. This also includes using mail.domain.net instead of mail.domain.local when connecting to OWA, OMA, RPC over HTTP and other published services. The probability is good (approaching 100%) that the VP won’t be happy with this situation and will demand the problem be fixed.

This situation is easily fixed by using a split DNS. With a split DNS infrastructure, you use the same name for internal and external access to resources. Instead of naming the internal network domain domain.local, you can use the name domain.net. There are no contraindications to using the same domain name for internal and external network access, and it simplifies things for your users by orders of magnitude.

Note that even if you already have an Active Directory domain in place that incorporates an invalid top level domain (such as the .local or .private), you can still use the split DNS infrastructure. All you need to do is create a DNS zone for the domain name you would like to use from the Internet. For example, even though the Active Directory domain name in this example is domain.local, you can still create a DNS zone on the internal DNS server for domain.net and enter the appropriate internal (private network addresses) Host (A) and MX records that allow internal network clients to connect directly to the Exchange Server on the internal network.

Using the split DNS infrastructure, we have two DNS servers: one of DNS server is configured to support internal network clients and the other DNS server is configured to support remote access clients on the Internet. The internal DNS server has a zone for the domain.net domain and has a Host (A) record for the Exchange Server that maps the name mail.domain.net to the IP address 10.0.0.2. The external DNS server also has a zone for the domain.net domain, but the external DNS server has a Host (A) record mapping the name mail.domain.net to 131.107.0.1.

Notice that both the internal and external DNS servers have a mapping for mail.internal.net. The only difference is that the internal DNS server maps the name to the actual IP address used by the Exchange Server, while the external DNS server maps the mail.internal.net name to the public IP address used in the ISA Server firewall’s Server Publishing Rule.

Now what happens to the VP who connects to the Exchange Server when on the internal network and when he travels to remote locations?

His email client application is configured to connect to the Exchange Server using the name mail.domain.net. When connected to the internal network his laptop computer resolves the name mail.domain.net to the Exchange Server’s internal address and directly connects to the Exchange Server via 10.0.0.2. When traveling to a customer location, he still connects to the Exchange Server using mail.domain.net, but this time connecting via IP address 131.107.0.1 -- the address on the external interface of the ISA Server firewall that you used in your Web and Server Publishing Rules to publish your Exchange Server’s services on the internal network.

The core of the split DNS includes:

  • Two physical DNS servers; you can not host the internal and external domains on the same server because the zone names are identical
  • The internal DNS server services clients connected to the internal network either via a direct Ethernet connection or via a VPN link
  • Two DNS zones with the same domain name; one zone contains the internal addresses required to make the direct connection to the Exchange Server’s services and the other zone contains the external or public addresses that are used to publish the Exchange Server’s services via ISA Web and Server Publishing Rules
  • The email client application is configured to use a fully qualified domain name to connect to the Exchange Server

You must make sure the email client application is assigned an address for a DNS server that can correctly resolve the name and that the client application is able to correctly resolve unqualified names correctly. The latter problem is specific for remote Outlook Exchange RPC access because of its continued dependence on NetBIOS names.

In most cases the email client computer obtains its IP addressing information via DHCP. The most common DHCP option assigned to DHCP clients is the DNS server option. You should configure the DHCP server on your internal network to assign the DHCP clients the DNS server address that can resolve the name of your split DNS domain.

For example, suppose your Active Directory domain name is domain.local. You can create a second DNS zone for domain.net on the same DNS servers that serve the domain.local domain, or you can put the domain.net domain zone on a separate DNS server. If you do put the domain.net zone on a separate DNS server, then you should create a DNS referral zone on the DNS servers serving the domain.local zone to forward requests for domain.net to the appropriate DNS servers on the internal network.

You do not have control over the DHCP servers on external networks. However, almost all remote networks assign via DHCP a DNS server that can resolve Internet host names. In this case, the email client on the remote network is assigned the address of a DNS server that can resolve Internet host names. The email client is configured to connect to the Exchange Server using the name mail.domain.net. The Internet DNS servers will resolve the name mail.domain.net to the public IP address on the external interface of the ISA Server firewall that you used in your Web or Server Publishing Rules.

The end result is that the email client does not need to be reconfigured based on network location. The split DNS allows the user to connect to the Exchange Server without ever needing to reconfigure the name of the Exchange Server. Email access just works.

Special Considerations for Secure SMTP/POP3/NNTP/IMAP4 and OWA SSL Connections

Your split DNS infrastructure supports access to all Exchange Server services. However, there are additional issues you need to consider when using SSL/TLS security on connections to the Exchange Server’s services.

The following requirements must be met to support SSL/TLS secured connections to the Exchange Server’s services:

  • The common name on the Web site certificate bound to the Exchange Server service must have a FQDN that can be resolved both on the internal and external network
  • The email client application must be configured to connect to the Exchange Server service using the common name on the certificate bound to the Exchange Server service
  • The email client must have the root CA certificate in its local machine certificate store, of the CA that issued the certificate to the Exchange Server’s service

The Common Name on the Web Site Certificate Must Be the Same as that Used by the Email Client

Secure Exchange email services have a Web site certificate bound to them before they can establish a secure link with an email client. The certificate bound to the service has a common name listed that identifies the secure mail service site. For example, when you request a Web site certificate for the SMTP service, you might have used the name mail.domain.net or smtp.domain.net or exchange.domain.net. Regardless of the name listed on the certificate, the email client machine must connect to the Exchange Server using that name.

Note:
The name on the certificate is determined by the person who requested the certificate. The Web site certificate Wizard on the Exchange Server allows you to enter the common name used by the service you bind the certificate to.

The Email Client Application must be Configured to Use the FQDN Listed in the Common Name on the Exchange Service’s Certificate

The email client application connecting to a secure Exchange Service must use the FQDN listed on the Web site certificate to connect to the service. For example, if you want to use a secure POP3 connection between the Outlook Express client and the POP3 service on the Exchange Server, the e-mail client must be configured to connect to the Exchange Server’s POP3 service using the name on the POP3 service’s certificate. If the certificate bound to the POP3 service on the Exchange Server has the common name pop.domain.net, then you need to configure the email client application to use the name pop.domain.net for the POP3 email server. You can not use the IP address of the POP3 server or the IP address on the external interface of the ISA Server firewall that is publishing the POP3 service via a Server Publishing Rule.

The reason why you must connect using the name listed on the certificate is that the client needs to confirm the identity of the server. The service identifies itself via the common name listed on the certificate. This process of the email service on the Exchange Server identifying itself is an important part of the security provided by secure SSL/TLS connections between the email client and server.

The Email Client Application must Trust the Certificate Presented by the Email Service

The secure mail service on the Exchange Server presents its server certificate to the email client as part of the security process that creates the secure channel between the email client and Exchange Server service. Part of this process includes the email client checking whether it trusts the certificate authority that issued the certificate to the secure Exchange Service.

The email client checks to see if the root CA in the list of CAs listed on the Exchange Service’s certificate is included in the client’s Trusted Root Certification Authorities certificate store. If the root CA certificate is not included in that list, the connection attempt will either fail, or the user will be presented with an error dialog box indicating that his machine does not trust the server. Neither of these situations is desirable and can lead to the inability to connect, or confusion on the email user’s part.

The problem is solved by importing the root CA certificate into the email client’s machine certificate store.

Discuss this article

Conclusion

In this article we discussed many of the common DNS issues that you must consider before deploying a remote access solution using ISA Server 2004. The properly configured DNS infrastructure, using a split DNS, will greatly enhance your users’ e-mail experience and reduce difficult to troubleshoot connectivity issues

Advertisement

Featured Links