DNS for ISA Server.

by Jim Harrison [Published on 11 Sept. 2001 / Last Updated on 20 May 2013]

Want some more fun? Let’s look at the ISA scenario. What many folks will do is place DNS resolver IPs in both NICs, ISP in the external, local in the internal. While this seems to make sense, it’s actually very inefficient and you can actually cause huge timeouts this way. Remember that TCP/IP will choose the route for a given packet based on its destination, not where it found the data. This means that DNS entries are not really NIC-specific, it’s just more meaningful to the person entering them.

Some history:

While Microsoft has made every effort to drop the NetBEUI protocol and the NetBIOS-over-TCP/IP support that legacy protocol required, reality had to set in somewhere. Legacy OS’s like W9x, WinME and even NT all use NetBIOS to a very large extent in their day-today operations. Since these OS’s still require functional networks in which to operate, and there must also be a method for upgrading the network from NetBIOS/WINS -based to DNS-based name resolution, NetBIOS/WINS name support still exists in W2K.

Definitions:

  • DNS - Domain Name Services; the services residing on a computer that are able to answer a name resolution query. How they answer that query depends on many settings for that server.
  • DDNS – Dynamic DNS; a new feature of DNS described in RFC 2136. It allows clients to update their own records in the DNS zone in which they’re listed. Details of a DDNS packet are in Q186346.
  • Forwarder – This is when one DNS server “forwards” name resolution of unknown names to another DNS server. Essentially, the forwarding server is acting as a DNS client with respect to the upstream DNS server.
  • FQDN – Fully Qualified Domain Name; this is a computer name that indicates its logical location by virtue of the domain structure associated with the name. For instance, www.microsoft.com indicates a host named “www” that lives in the domain “microsoft” under the top-level domain “com”. These names are always separated by dots “.”.
  • NetBEUI – NetBIOS Extended User Interface; the protocol introduced in Windows NT3 and W3x. Unlike TCP/IP, this is a non-routable protocol that is useless outside a single subnet.
    NetBIOS – Network Basic Input-Output System; this is the basis of the NetBEUI protocol
  • Record – This is an entry in a DNS zone that represents a single element, such as a host, a mail server or even another zone.
  • TTL – Time-to-live; indicates how long (in seconds) a name record may live in the requester’s name cache before it must be re-requested again
  • WINS – Windows Internet Name Services; this is a name resolution service similar to DNS, except that it deals strictly with NetBIOS names

W2K Name Resolution:

There are two client services on each and every Microsoft OS that are capable of resolving computer names:

  • DNS cache – this is the local name cache for computer names that have been resolved via queries to DNS servers.  Since it resides in memory, this is the first choice for resolution of fully qualified domain names, or FQDN.  Resolved names are kept until the respective TTL expires, then they are flushed.  You can also flush this cache with ipconfig /flushdns.
  • WINS client – this is the local service for resolving unqualified names via queries toWINS servers.  The name-to-IP mapping that results from these queries is stored in the NetBIOS name cache.  This is also the service that will resort to NetBIOS name queries over UDP ports 137 and 138.  This cache can be cleared with an nbtstat –R command.In addition, there are two files that work in conjunction with the name resolution client services on every Microsoft OS:
  • Hosts file – this is a text file containing FQDN-to-IP mapping.  It operates much like a DNS server to the DNS cache service, but has no TTL associated with the names it maps.  Consequently, the names resolved here live in the DNS cache until the cache is flushed.
  • LMHosts file – this is a text file that contains NetBIOS name-to-IP mapping.  It operates much like a WINS server, resolving unqualified names.Generally speaking, if you put all these name resolution methods together in their W2K-preferred usage order, they would operate like this:
  • Fully Qualified Domain Names (FQDN)
    1. DNS name cache
    2. Hosts file
    3. Query a DNS server
  • Unqualified (NetBIOS) names
    1. NetBIOS name cache
    2. LMHosts file
    3. Query a WINS server
    4. NetBIOS name broadcast

You'll notice that the name resolvers try to stay on the local machine as much as possible.  This is because if the name can be resolved locally, we can avoid any potential networking issues that may slow the whole process.  Also, you should be aware that prior to W2K, all Microsoft OS's resolved names using unqualified methods first.

Want some more fun?  Let's look at the ISA scenario.  What many folks will do is place DNS resolver IPs in both NICs, ISP in the external, local in the internal.  While this seems to make sense, it's actually very inefficient and you can actually cause huge timeouts this way.  Remember that TCP/IP will choose the route for a given packet based on its destination, not where it found the data.  This means that DNS entries are not really NIC-specific, it's just more meaningful to the person entering them.  W2K uses each DNS resolver in this fashion:

  1. NIC1, DNS1
  2. NIC2, DNS1
  3. NIC1, DNS2
  4. NIC2, DNS2

..and so on down the DNS list.  Add the whole DNS suffix search list to each DNS query, (can be huge in large deployments or those with multi-level domain names), and you have a potential DNS disaster of gargantuan proportions.  Another feature of W2K DNS resolver usage is that if one DNS server in the list of a given interface fails to respond (not an "I don't know" answer; that's different), then that whole NIC is blacklisted from the DNS search.  So if DNS2 on NIC1 fails to respond to a query, then DNS1 on that interface will also be ignored for a while.  For those reasons, it's best to place all DNS resolver IPs in the internal ISA NIC.

Now let's add even more fun to the game; Windows (all flavors) can use DNS to resolve NetBIOS names.  What, you say?  Didn't I just say that DNS resolved fully qualified names?  Yes, I did say that.  What I haven't fully discussed yet is a neat feature called the "suffix search list".  This is a list of domain "suffixes" that can be appended to a NetBIOS name to help resolve it via DNS.  Why would we want to do this?  Well, say you have an internal AD-based domain called "mydomain.com" and you want to ping your ISA server from a client to test name resolution.  You could type ping isa.mydomain.com, but your fingers are just a little tired from all the typing today, so you just type ping isa.  Guess what; if you've set up the domain properly, you get: Pinging isa.mydomain.com [192.168.0.1] with 32 bytes of data: for your efforts.  How is that possible, you ask (go ahead; ask)?  This is the result of the DNS cache service using the suffix search list that is provided by the AD when you join a domain.  If you were to open a command window and type ipconfig /all, you'd likely see something like this:

Host Name : client
Primary DNS Suffix : mydomain.com
Node Type : Hybrid
IP Routing Enabled : No
WINS Proxy Enabled : No
DNS Suffix Search List : mydomain.com

What the DNS cache service is doing is appending "mydomain.com" from the suffix search list to "isa" to create "isa.mydomain.com" and passing that to the DNS server for name resolution.  This way, we stay in the DNS name resolution stage instead of dropping to WINS / NetBIOS broadcasts.  Neat trick, huh?  Obviously, this doesn't work too well for Internet names, but then it wasn't designed to.  It's only to make internal name resolution more zippily.

Wait, what's that you say; you don't have an AD structure in your LAN?  Not to worry; you can add your own search list either manually (for static IP's) or via DHCP settings and we'll cover both of those.

Static IP

    1. R-click My Network Neighborhood, select Properties

    2. R-click Local Area Connection and select Properties

    3. Double-click on Internet Protocol (TCP/IP) or select it and click Properties

    4. Click the Advanced button, then the DNS tab

    5. Select Append these DNS suffixes (in order) and click the Add button

    6. Enter the domain name in the box provided and click Add

    7. You will now have the domain name listed in the suffix search list as shown below.

    8. Click OK three times to close all open properties windows; now you have a suffix search list, but only for this computer.  You'll have to repeat this procedure on all the machines in your LAN.

DHCP-assigned IP (at the DHCP server, of course)

    1. R-click My Computer and select Manage

    2. Expand Services, DHCP, then Scope

    3. R-click Scope Options, select Configure Options

    4. Scroll down the list until you see 015 - DNS Domain Name

    5. Select that option and enter your domain name (not host name) in the String value box; click Apply, then OK

    6. Perform an ipcponfig /renew at each DHCP client to make sure they get the new setting

ISA DNS behavior:

ISA web proxy service also provides for a DNS "proxy" above and beyond any other DNS services that exist on the ISA server.  This proxy is only available to WEB and Firewall clients, not SecureNAT clients.  Bear in mind that the DNS proxy maintains a 6-hour TTL for all records, regardless of the actual record TTL.  This means that any name resolved by the WEB proxy service will remain in the ISA DNS proxy cache for six hours, or until you restart the web proxy service.  The good news is that if the client has a DNS resolver IP in its IP settings, they'll use that first, potentially avoiding the ISA 6-hour DNS cache.

Things to consider when deciding how to set up your DNS solution:

1.      Active Directory domains require DNS
2.      ISA Array performance depends heavily on access to the AD schema
3.      ISA server array participation is heavily dependent on inter-ISA communications
4.      W2K DNS resolution is attempted before NetBIOS / WINS
5.      The DNS server search process includes all the configured interfaces in multi-homed servers (ISA, for instance)
6.      More DNS server IP's on each server = greater chance of timeout errors
7.      What internal services will require DNS support (AD, SMTP, POP, etc.)?
8.      What external services will require DNS support (SMTP. WEB, FTP, etc.)?

DNS services are needed, period.  What kind of DNS solution fits you best is something only you can decide.  DNS services can co-exist with ISA, but the setup isn't as simple as when the DNS server is a SNAT client.

The following table will help you define your DNS requirements:

Question

ISP DNS Server(s)

Your DNS Server(s)

DNS Forwarding

DNS Publishing

External Zone

Internal Zone

Will you be publishing internal servers?

Not advised

Preferred

Not Required

Preferred

Required

Preferred

Will you be using ISA Arrays?

Not useful

Required (AD)

Not Required

Not Required

Not Required

Required (AD)

Are you running over Dial-up?

Advised

Preferred

Not Required

Not Advised

Not Advised

Preferred

Do you need to keep your internal DNS private?

Not Required

Required

Required

Separate DNS Server

Separate DNS Server

Required

NOTES

  • You’ll notice that most of the scenarios presented can be combined to further complicate matters and expand the flexibility of your setup.  Choose your requirements carefully, as some scenarios make others nearly impossible, like publishing over dial-up.
  • You’ll notice that many of the machines running DNS server services have “127.0.0.1” as its preferred DNS resolver.  I’ve recommended this because if you’re using DDNS (W2K does, by default), then generally speaking, you want the DNS server to update its own zone, not another server.  The exception to this is when the DNS server is not primary for that zone. 
  • You can’t enter 127.0.0.1 manually in TCP/IP settings as shown in some of the drawings, but if you delete all DNS resolver IPs from the list on a server running DNS services, W2K will notify you that you’ve left the entries blank and it will kindly enter “the local IP address” for you.  In fact, what it does is enter 127.0.0.1 in the Preferred DNS server.
  • DNS Name Queries use UDP port 53 to the DNS server, while DNS Zone Transfers use TCP port 53 to the server.  In both cases, the client port is dynamic.
  • I’ve summarized the Packet Filter and Protocol Rule settings for each scenario; detailed procedures for each are contained in the ISA help.  It’s a good document; read it.

CAVEATS

  • DNS is unforgiving if not installed and configured correctly.  Read the W2K DNS help before installing it on your server.
  • DNS on the same server with AD needs some extra attention.  You need to make the NetLogon and DNSCache services dependent on the DNS server service to avoid AD startup errors.  Q193888 tells you how to do that.  It takes two reboots for this change to take effect.
  • If you’re running a Windows-based SMTP server (IIS5 or Exchange), you’ll need to add TCP-53 to the packet filter and / or protocol list if it isn’t already included.  See Q292278 for details.
  • Do not enable DDNS for your external zones unless you want to allow the bad guys out there to mess with your zone data.  The difference between a DNS query and a DNS update is a bit pattern in the DNS protocol header.  Both requests are made via UDP port 53, so they both look like queries to ISA and can’t be blocked on a simple protocol / port basis.
  • I’ve made the assumption that you’ve used the instructions for setting up your ISA server from the isaserver.org Learning Zone tutorials
  • The following drawings indicate DNS traffic paths only, not actual network connections

Drawing 1

Drawing 1 is what you might use if you only wanted Internet name resolution (in a dial-up scenario, for example).  Note that the clients including the ISA server have the ISP DNS server as their preferred DNS.  You can publish servers using this setup, but it requires the cooperation of your ISP.  Generally speaking, if they offer hosting services, you're good to go.  The disadvantages are that you have to email or call them to get your zone data changed and you can't add your internal IPs to that zone, so it can't serve your internal network.

ISA Setup:

1. Packet Filters:
- DNS Lookup (UDP-53) out

2. Protocol Rules:
-
DNS Lookup (UDP-53) out

3. DNS Publishing:
-
Not generally available


Drawing 2

Drawing 2 shows another single-DNS server solution, using the ISA server.  While this seems like an easily implemented scenario, be aware that it's quite different both in concept and setup.  You'll notice that the ISA has "127.0.0.1" as its preferred DNS resolver.  This is common for DNS server setup; it tells the server's DNS client service to use the local server for DNS resolution.  Although the ISP DNS server isn't shown here, you can still make use of it as a forwarder or just a second DNS server.

ISA Setup:

1. Packet Filters:
-DNS Lookup (UDP-53) out / in
-DNS Zone Transfer (TCP-53) out / in

2. Protocol Rules:
-
DNS Lookup (UDP-53) out

3. DNS Publishing:
- Via inbound packet filters

Drawing 3

Drawing 3 depicts a somewhat more complex, but more flexible scenario; the use of separate internal and external DNS resolvers.  It allows you to separate the functions of the internal and external DNS zones, while still making them both available to the clients.    The "Client DNS#" paths indicate which logical path the client would take to use a given DNS server.  You can choose to limit the client DNS resolvers to the internal DNS server by using the external DNS server as a "forwarder".  This simplifies client setup, but still resolves external names easily.  Either way, the ISA setup is the same.

ISA Setup:

1. Packet Filters:
- DNS Lookup (UDP-53) out

2. Protocol Rules:
-
DNS Lookup (UDP-53) out

3. DNS Publishing:
-
DNS server publishing rule using DNS Query to INT DNS


Drawing 4

Drawing 4 depicts separate internal DNS servers with redundancy.  This is the scenario you might choose for a W2K AD environment.  DNS and AD coexist easily enough, so that your primary and backup ADs can also serve as your primary and secondary DNS servers.  Make your internal zone AD-integrated, and there's no such thing as primary or secondary DNS zones.

ISA Setup:

1. Packet Filters:
-
DNS Lookup (UDP-53) out

2. Protocol Rules:
-
DNS Lookup (UDP-53) out

3. DNS Publishing:
- DNS server publishing rule using DNS Query to INT DNS01 and / or INT DNS02


Drawing 5

Drawing 5 shows the last variation on a theme, where the DNS server hosting the external zones and the DNS server hosting the internal zone are both located behind the ISA server.  This is the one for the seriously paranoid (or just severely burned) among us.  In this setup, the internal DNS is totally invisible to the Internet, because it can't resolve Internet names without forwarding to EXT DNS.  All other setups allow the internal-zone DNS server access to the Internet via ISA.

ISA Setup:

1. Packet Filters:
- DNS Lookup (UDP-53) out

2. Protocol Rules:
-
DNS Lookup (UDP-53) out

3. DNS Publishing:
-
DNS server publishing rule using DNS Query to EXT DNS

See Also


The Author — Jim Harrison

Jim Harrison, MCP(NT4/W2K), A+, Network+, ET1(SW)(RET), is a husband to one and father to four. He's been active in online computing ever since he built his first bulletin board on a Kaypro-4 (I miss that tank).

Advertisement

Featured Links