Networks are under constant attack from Internet intruders. Hackers can use a variety of methods to compromise the security of your firewall. Internet based attackers have the advantage of being able to work on a planned attack against your network over days, weeks or months. During that interval, Internet attackers leave a trail of evidence that can be used to prevent a successful attack and successfully prosecute the attacker. Firewalls need to detect and prevent both network and application layer attacks, such as those based on IP packet fragmenting and DNS protocol exploits. ISA Server 2000 firewalls come with a collection of network and application layer intrusion detection filters that can detect and block common attacks. These intrusion detection filters can also be configured to log the attacks in the firewall’s Event log and send a message to the security administrator.

In this document we will discuss each of these issues and how ISA Server 2000 solves them. This document also includes step by step examples on how to configure the ISA Server 2000 firewall to detect common attacks and report them, how to configure the packet filter logs and finally, how to create a packet filter to allow selective reporting for blocked packets.

The Problem: Detecting and Blocking Attacks Against the ISA Server 2000 Firewall

Firewalls are under constant attack from Internet intruders. The firewall at the edge of the network is the first to receive attack packets from Internet based attackers. These attacks continue around the clock because of the worldwide and “always-on” availability of corporate Internet connections.

Internet intruders are using increasingly complex and effective methods to attack the front-end firewall and gain access to corporate network resources. The Internet-based attackers typically have the advantage of time; they can continue to test your firewall defenses around the clock until they find an effective method to overcome firewall functionality.

Attackers often spend days, weeks or months reconnoitering your firewall security infrastructure before they are able to mount a successful attack. The Internet intruder typically tries a variety of techniques before finding the one that compromises firewall protection. If you can find a method of detecting the attacker’s early attempts at defeating your defenses, then you can take proactive countermeasures and report the attacker’s activities to the proper law enforcement authorities.

The firewall needs to be able to detect application and network layer attacks. There are a number of network layer attack, such as those based on fragmented IP packets and IP Options that allow the attacker to change the routing behavior on the firewall.

Traditional packet filtering firewalls can detect network layer attacks but are unable to effectively detect application layer attacks because the packet filtering firewall does not understand the application layer protocols. Application layer aware firewalls have the ability to detect an application layer attack and report their findings.

The Solution: ISA Server 2000 Intrusion Detection, Fragment Filtering, Options Filtering and Packet Filter Logging

ISA Server 2000 firewalls provide powerful solutions to each of these problems. The ISA Server 2000 firewall is able to detect application layer and network layer intrusions, detect and block IP fragment and IP Options based attacks, log all packets moving through the ISA Server 2000 firewall and alert you to attack conditions so that you can respond in real time.

All of these goals are met by the following ISA Server 2000 firewall features:

ISA Server 2000 intrusion detection filters
IP fragment filtering
Options filtering
Packet filter logging

ISA Server 2000 Intrusion Detection

The built-in intrusion detection system used by ISA Server 2000 detects two general types of attacks:

Application layer intrusion detection filters
Network layer intrusion detection filters

Application layer attacks take advantage of potential flaws in the server service that is being attacked. For example, an attacker may use a buffer overflow attack in the application layer commands for the SMTP or DNS service. Detection and prevention of application layer attacks requires that the firewall have an understanding of the application layer protocols.

Network layer attacks are aimed at lower levels of the TCP/IP protocol “stack”. Examples of network layer attacks include the ping of death attack and the IP half scan. The goal of network layer attacks is to disable the networking capabilities of the firewall or server being attacked. This creates a denial of service condition by removing the server from the network, either because the networking component of the server no longer functions, or because the entire operating system is disabled.

Application Layer Intrusion Detection Filters

ISA Server 2000 has two built-in application layer intrusion detection filters:

The DNS intrusion detection application layer filter
The POP intrusion detection application layer filter

DNS intrusion detection application layer filter

The DNS intrusion detection filter works together with DNS Server Publishing rules. The filter intercepts and analyzes all inbound DNS traffic destined for the internal network. You can configure the filter to check for the following intrusion attempts:

DNS hostname overflow
DNS length overflow
DNS zone transfer from privileged ports (1–1024)
DNS zone transfer from high ports (above 1024)

DNS hostname overflow

A DNS hostname overflow occurs when a DNS response for a host name exceeds a certain fixed length. Applications that do not check the length of the host names may overflow internal buffers when copying this host name, allowing a remote attacker to execute arbitrary commands on a targeted computer.

DNS length overflow

DNS responses for IP addresses contain a length field, which should be four bytes. By formatting a DNS response with a larger value, some applications executing DNS lookups will overflow internal buffers, allowing a remote attacker to execute arbitrary commands on a targeted computer.

DNS zone transfer from privileged ports (1–1024)

A DNS zone transfer from privileged ports (1–1024) occurs when a client system uses a DNS client application to transfer zones from an internal DNS server. The source port number is a privileged port number (between 1 and 1024), indicating a client process. A zone transfer is not necessarily an attack. For example, external DNS servers in your perimeter network may need to perform zone transfers from source port UDP 53.

DNS zone transfer from high ports (above 1024)

A DNS zone transfer from high ports (above 1024) occurs when a client system uses a DNS client application to transfer zones from an internal DNS server. The source port number is a privileged port number (between 1 and 1024), indicating a client process. A DNS zone transfer from a high port indicates that an attacker may be gathering information about the resources in your domain by learning about hosts listed in your resource records.

POP intrusion detection application layer filter

The Post Office Protocol (POP) intrusion detection filter works together with POP3 Server Publishing Rules to intercept and analyze POP3 traffic destined for the internal network. The POP3 application filter checks for POP3 buffer overflow attacks. A POP3 buffer overflow attack occurs when a remote attacker attempts to gain control of a POP3 server by overflowing an internal buffer on the server.

Network Layer Intrusion Detection

ISA Server 2000 can detect a number of network layer intrusions. These include:

All ports scan attack
Well-known port scan attack
IP half scan attack
Land attack
Ping of death attack
UDP bomb attack
Windows out-of-band attack

All ports scan attack

This alert notifies you that an attempt was made to access more than the preconfigured number of ports. You can specify a threshold, indicating the number of ports that can be accessed without triggering this alert.

Enumerated port scan attack

This alert notifies you that an attempt was made to count services running on a computer by probing each port for a response. If this attack is detected, you should check the packet filter logs and identify the source of the port scan. Compare your findings from the packet filter log analysis with services running on the target computer. Check the packet filter, Web Proxy and Firewall service logs for indications of unauthorized access. You should consider the system compromised and take appropriate action if you detect indications of unauthorized access,

Port scanning, in itself, does no harm to your network or system, but it provides hackers with information they can use to penetrate the network.

IP half scan attack

This attack takes place when repeated attempts to connect to a destination computer are made with no corresponding ACK packets. A standard TCP connection is established by sending a SYN packet to the destination computer. If the destination is waiting for a connection on the specified port, it responds with a SYN/ACK packet. The connection is established when the initial sender replies with an ACK packet. If the destination computer is not waiting for a connection on the specified port that it receives the ACK packet, it responds with an RST packet.

Most system logs do not log completed connections until the final ACK packet is received from machine that sent the initial SYN packet. Sending an RST packet instead of the final ACK results in the connection never being established. The connection is not logged because the connection attempt was never fully completed. Because the initial sender can identify whether the destination sent a SYN/ACK or RST packet, an attacker can determine which ports are open for connections, without the destination being aware of the probing.

Log the address from which the scan occurs. If appropriate, configure the ISA Server policy rules or Internet Protocol (IP) packet filters to block traffic from the source of the scans if the ISA Server 2000 firewall detects this attack. You should also consider informing the ISP of the attacker about the attack and forwarding packet filter logs to the ISP when they are requested.

Land attack

This attack takes place when a TCP SYN packet is sent with a spoofed source IP address and port number that matches the destination IP address and port. The Land attack can cause some TCP/IP protocol implementations to go into a loop that crashes the computer. Configure ISA Server 2000 IP packet filters to inhibit traffic from the source of the scans if this alert occurs.

Ping of death attack

This attack takes place when a large amount of information is appended to an Internet Control Message Protocol (ICMP) echo request (ping) packet. When the computer attempts to respond, a kernel buffer overflow can result and crash the computer. Create a packet filter that specifically denies incoming ICMP echo request packets from the Internet If this alert occurs,

UDP bomb attack

This attack takes place when someone sends an illegal User Datagram Protocol (UDP) packet. A UDP packet has illegal values in certain fields, and this causes some older operating systems to crash. It is often difficult to determine the cause if the target machine crashes.

Windows out-of-band attack (WinNuke)

The out-of-band (OOB) attack exploits a vulnerability in Microsoft networks. The WinNuke program (and variations such as Sinnerz and Muerte) creates an out-of-band data transmission that crashes the victim machine. A TCP/IP connection is established with the target IP address, using port 139 (the NetBIOS port). Then the program sends data using a flag called MSG_OOB (or Urgent) in the packet header. This flag instructs Windows sockets (Winsock) to send data called out of band data.

Upon receipt, the targeted Windows server expects a pointer to the position in the packet where the Urgent data ends, with normal data following, but the OOB pointer in the packet created by WinNuke points to the end of the frame with no data following. The machine does not know how to handle this and ceases communicating on the network. A WinNuke attack usually requires a reboot of the affected system to reestablish network communications.

ISA Server 2000 Fragment Filtering

All fragmented packets are dropped when ISA Server 2000 filters packet fragments. The “teardrop” attack and its variants involve sending fragmented packets and then reassembling them in such a way that may cause harm to the system. The teardrop attack works a little differently from the Ping of Death, but with similar results. The teardrop program creates IP fragments, which are pieces of an IP packet into which an original packet can be divided as it travels through the Internet. The problem is that the offset fields on these fragments, which are supposed to indicate the portion (in bytes) of the original packet that is contained in the fragment, overlap.

For example, normally two fragments’ offset fields might appear as shown below:

Fragment 1: (offset) 100 – 300

Fragment 2: (offset) 301 – 600

This indicates that the first fragment contains bytes 100 through 300 of the original packet, and the second fragment contains bytes 301 through 600.

Overlapping offset fields would appear something like this:

Fragment 1: (offset) 100 – 300

Fragment 2: (offset) 200 – 400

When the destination computer tries to reassemble these packets, it is unable to do so and may crash, hang or reboot.

Fragment filtering can interfere with streaming audio and video. In addition, L2TP/IPSec connections may not be successfully established because packet fragmentation may take place during certificate exchange. Disable fragment filtering if you have problems with streaming media and IPSec based VPN connections.

ISA Server 2000 Options Filtering

You can configure ISA Server to refuse all packets that have the words "IP Options" in the header. The most problematic options are the source routing options. TCP/IP supports source routing, which is a means to permit the sender of network data to route the packets through a specific point on the network. There are two types of source routing:

· Strict source routing: the sender of the data can specify the exact route (rarely used).

· Loose source record route (LSRR): the sender can specify certain routers (hops) through which the packet must pass.

The source route option in the IP header allows the sender to override routing decisions which are normally made by the routers between the source and destination machines. Network administrators can use source routing to map the network or to troubleshoot routing and communications problems. Source routing can also be used to force traffic through a route providing the best performance. Unfortunately, source routing can be exploited by attackers.

An intruder can use source routing to reach private internal addresses on the LAN that normally are not reachable from the Internet by routing the traffic through another machine that is reachable from both the Internet and the internal machine

ISA Server 2000 Packet Filter Logging

All packets passing through the ISA Server 2000 firewall can be logged to the packet filter log. You can configure which packets are logged:

All dropped packets are logged to the packet filter log by default. Logging is turned off when you disable packet filtering.
You can configure the ISA Server 2000 firewall to disable logging for packets dropped due to deny IP packet filters.
You can configure the ISA Server 2000 firewall to log all packets received at the external interface. All packets passing through ISA Server are logged in the packet filter log when you enable logging of allowed packets.

Logging allowed packets and blocked packets causes a considerable processing load on the server and consumes large amounts of disk space.

Packet filter event messages

These messages refer to events connected to packet filtering in Microsoft Internet Security and Acceleration (ISA) Server.

Message ID	Description
14038	ISA Server packet filter log service could not allocate memory.
14039	ISA Server packet filter logging component cannot obtain the log contents.
14043	Could not create the system-wide packet filter log event.
14044	The packet filter is dropping IP packets.
14046	Packet filter protocol violation.
14086	Insecure configuration detected.
14119	An external interface could not be found for packet filtering.
14120	The ISA Server services could not create a packet filter %1.
14121	The packet filter dial-out interface cannot be rebound.
14122	A packet filter interface could not be bound.
14123	Failed to create the IP packet filter.
14124	Filtering disabled as requested.

Field position	Descriptive name (field name)	Description
1	Date (date)	Date the packet was received.
2	Time (time)	The time the packet was received (service info fields)
3	Source IP (source-ip)	The Internet Protocol (IP) address of the source (remote) computer. The source computer is the computer from which the data packets originated.
4	Destination IP (destination-ip)	The IP address of the destination (local) computer. The destination computer is usually the ISA Server computer.
5	Protocol (protocol)	The particular transport level protocol that is used during the connection, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or Internet Control Message Protocol (ICMP).
6	Source port (or protocol type, if ICMP) (param#1)	For TCP and UDP protocols, the remote port used to create a connection. For ICMP protocol, the type used when creating the connection.
7	Destination port (or protocol code, if ICMP) (param#2)	For TCP and UDP protocols, the local port used to create a connection. For ICMP protocol, the code used when creating the connection.
8	TCP flags (tcp-flags)	For a TCP data packet, represents the TCP flag value in the IP header. The possible values are FIN, SYN, RST, PSH, ACK, and URG.
9	Interface (filter-rule)	Indicates whether the packet was accepted (1) or dropped (0). By default, only dropped packets are logged.
10	Interface IP address (interface)	Interface on which the packet was received; usually only one interface.
11	Header (ip-header)	The entire IP header of the data packet that generated the alert event. The IP header is logged in hexadecimal format.
12	Payload (payload)