Save $400 on 4-6 day Courses at SANS Cyber Defense Initiative 2017. Ends Tomorrow!

Malware FAQ

Malware FAQ: Checkpoint Firewall-1 ACK DoS vulnerability and sample incident

Author: Brian Carlson

On Friday the 23 rd of February 2001 around 4:30 we received a call from the LAN staff at Smaller Company. I work for Larger Company in our east coast offices. Larger Company is a competitive local exchange carrier (CLEC) as well as a cable television and Internet company. We had recently acquired Smaller Company in the Midwest as a matter of growth. Smaller Company offers the same services and proved to be a good fit.

The LAN staffer explained that they were having problems reaching the Internet. Larger Company has a centralized corporate network security function. We were in the process of integrating Smaller Company with Larger Company and had replaced their firewall with one of our standard configuration. During the transition we had received many calls of this sort. Surfing is not a business critical function but there were Back Office applications that interfaced with hosts outside our perimeter firewalls across the same link. We proceeded to investigate.

Due to difficulties merging companies, Smaller Company's firewall did not have as restrictive a policy as we require for all our other firewalls. In the interest of business continuity we had not disabled the existing rules that were not completely accounted for. Over time we would implement egress filtering and remove and restrict the inbound traffic.

We attempted to log into the perimeter firewall, Checkpoint Firewall-1 4.1 Service Pack 1 running on a Sun Ultra 5 300 Mhz processor with 512 Megabytes of memory. The large amount of memory was present on the system to support network analysis through Ntop and periodic use of the firewall as a sniffer via Solaris' Snoop utility and Tcpdump .

There was no response from our ssh connection attempts for a significant period of time. When the password prompt finally appeared we experienced further delay logging in. When responsive interactive shells finally appeared we checked system load (the number of jobs in the system run queue over the last 1, 5 and 15 minutes). Our system had values of 14, 23 and 38. At the time we were accustomed to seeing values in the vicinity of 0.3. We had a problem.


The Exploit software used in this incident appears to have been Nmap . If you are not familiar with Nmap it is a network mapping and scanning utility by implemented using libpcap. Nmap is a marvelously flexible scanner. It is capable of OS fingerprinting by system responses to uniquely crafted packets. It can perform a wide variety of scans with varying IP flags enabled. Nmap can fragment the packets it generates. These and its many other features allow it to operate with different levels of aggression and under different levels of stealth.

Nmap is not an exploit in a root compromise sense, nor is it a denial of service tool by design. It is however amazingly flexible

The two vulnerabilities specifically exploited within this incident were:

1) The Checkpoint Firewall-1 ACK DoS Vulnerability


Under the TCP/IP protocol a session is established between two hosts through a process commonly referred to as the three-way handshake. The originating host requests the establishment of a session with the destination host. The destination host acknowledges/accepts the request for a connected session while at the same time requesting a session of its own with the originating host. When the originating host acknowledges/accepts that request from the destination host the session is considered established.

Once a session is established data can be sent between the two hosts. If data is being placed in the packet for the application on the other end of the connection the push flag is set when the packet is sent along with an acknowledgement of the previous packet sent in the session. Upon receipt the destination host replies with an acknowledgement of the data received. This exchange continues with either participating host able to initiate transmission of data. The lifetime of this session between the two hosts is dictated at the application level by the application driving the exchange. Extended periods of time may pass without either host transmitting data.

Checkpoint Firewall-1 tracks all established sessions for a finite period of time. When a TCP/IP session ends naturally (by a fin negotiation or a reset sent by one of the hosts) the connection table entry is removed. If a connection table entry is still present after the defined ¡§idle timeout¡¨ has elapsed the connection table entry is removed. Checkpoint Firewall-1 presumes the session has ended or become invalid in some manner (i.e. a participating host has crashed or become disconnected). The idle timeout is necessary to prevent resource consumption from negatively affecting the performance of the firewall.

Checkpoint Firewall-1 does attempt to provide a remedy for sessions that have been potentially incorrectly invalidated. A new connection table entry is created while awaiting confirmation or denial of an existing session by the destination host. If the destination host is unresponsive (i.e. no longer exists or is disconnected from the network) the connection table entry is retained until the idle timeout expires the pending connection. As a result a flood of ACK initiated sessions can fill the connection table with bogus entries resulting in a denial of service attack.

The firewall steps outside TCP/IP by invalidating connections at the security enforcement point. By then attempting to re-validate or re-establish potentially valid sessions based only on a packet containing an acknowledgement flag Checkpoint Firewall-1 leaves itself open to exploitation.

Affected Systems:

This vulnerability was present in all implementations (AIX/RS6000, HP-UX/PA, Solaris/SPARC, Solaris/x86, Windows NT/x86) of Checkpoint Firewall-1 4.0 until service pack 7 and in Checkpoint Firewall-1 4.1 until service pack 2.

Checkpoint released an Inspect Code modification to change the behavior of previous revisions of the firewalls. This change was made standard in the aforementioned service packs.

Additional information can be found at:

In the course of my research I have not found a tool designed specifically to exploit this vulnerability, i.e. stream ACK packets for the express purpose of creating a denial of service attack on a Checkpoint Firewall-1 firewall. Other more generic tools exist that provide this capability. Specifically, Nmap configured to perform a scan based on ACK packets affects the vulnerability.

2) The Checkpoint Firewall-1 IP Fragment-driven Denial of Service Vulnerability


Under the IP layer of the TCP/IP protocol suite packets are sent across physical network links. before a packet can be placed on a physical link the interface to the physical network is queried to determine the size of packet it is capable of handling. If the TCP/IP packet is too large to be supported by the physical network the packet is fragmented into pieces the physical network can handle. These packets are not reassembled until they reach their destination. All the information required for reassembling the original packet is contained within the IP header.

Checkpoint Firewall-1 evaluates the entire TCP/IP packet when comparing it against its table of existing connections and the installed firewall policy. To perform this action it must reassemble the fragmented packets. To adhere to the RFC it must then re-fragment the packet in the same manner as the originating host before forwarding them. This virtual de-fragmentation/re-fragmentation process and issues with the way in which Checkpoint Firewall-1 logs these packets can lead to consumption of CPU resources. If the firewall receives a large volume of fragmented packets a denial of service can result.

Affected Systems:

This vulnerability was present, in its initial form, on all implementations (AIX/RS6000, HP-UX/PA, Solaris/SPARC, Solaris/x86, Windows NT/x86) of Checkpoint Firewall-1 4.0 until the release of service pack 6. The vulnerability was also present in Checkpoint Firewall-1 4.1 until service pack 2. Post these service packs the software was modified to provide tunable parameters to mitigate the affect {JWL effect?} of this kind of attack.

Additional Information can be found at:

We determined that Nmap was most likely the tool used in the fragment-driven denial of service attack we experienced. There is an additional tool commonly referenced that could have also been responsible. Most commonly associated to DoS attacks against windows, Jolt2 can be used to the same affect against a Checkpoint Firewall-1. Jolt2 sends a stream of malformed IP fragments against a target.

While not intended as a denial of service tool, Fragrouter converts any source of network traffic into a packet stream capable of affecting an attack on a vulnerable Checkpoint Firewall-1 firewall. Fragrouter was designed as a utility for testing the effectiveness of Intrusion Detection Systems. From the README, ¡§Conceptually, fragrouter is just a one-way fragmenting router ¡V IP packets get sent from the attacker to the fragrouter, which transforms them into a fragmented data stream to forward to the victim.¡¨

An Intrusion Detection Systems has to reassemble an entire IP packet to be able to evaluate the impact of the whole packet. Breaking the packet into fragments obscures malformations and malicious content. Tracking fragments for reassembly negatively impacts the performance of the IDS system and may not be performed.


These two previously mentioned CVEs directly reference the weaknesses within the implementation of Checkpoint Firewall-1 exploited to deny us service. It is worthwhile to note that both fall under a proposed CVE for the greater class of ¡§Naptha¡¨ denial of service attacks. The following is excerpted directly from the CVE.

CAN-2000-1039 (under review)

¡§Various TCP/IP stacks and network applications allow remote attackers to cause a denial of service by flooding a target host with TCP connection attempts and completing the TCP/IP handshake without maintaining the connection state on the attacker host, aka the "NAPTHA" class of vulnerabilities. NOTE: this candidate may change significantly as the security community discusses the technical nature of NAPTHA and learns more about the affected applications. This candidate is at a higher level of abstraction than is typical for CVE.¡¨

The Attack:

In order to understand how the application fails you need to first understand how sessions are established under the TCP/IP protocol suite. Within TCP/IP there are 4 protocol layers each responsible for different aspects of communication; Link, Network, Transport and Application.

The Link layer takes responsibility for moving data across the physical hardware. Device drivers and network interfaces reside at this level. Different link types potentially have different capacities for packet size.

Above the Link layer the Network layer takes responsibility for the movement of packets around the network and enables movement across hardware boundaries. IP, ICMP and IGMP are examples of protocols within this level. If a packet's size exceeds the maximum capacity of the link layer, the MTU (maximum transmission unit) value for that link, IP breaks the packet up, or fragments it. Each of the fragment packets are constructed to be smaller than the MTU so that they may be transmitted upon the link. These fragmented packets are not reassembled until the entire set has arrived at the destination host. Having intermediate devices assemble and re-fragment packets would be a waste of resources. The packets may take differing paths to their ultimate destination such that intermediate devices may not ever acquire the entire set fragment packets required to recreate the original packet.

Above the Network layer resides the Transport layer. In the case of TCP, Transmission Control Protocol, it provides Transmission Control. Capabilities are provided to support acknowledgement of received packets and timers are present to attempt to determine failure of receipt. TCP provides reliable delivery for connected sessions. Also at the transport layer UDP, User Datagram Protocol, operates in a fire and forget fashion. Packets are sent without regard for successful delivery. Responsibility for ensuring data has been transmitted is pushed up to the next layer in the protocol suite, the Application layer.

The Application layer contains data specific to the associated application. The content of that data is entirely dependant upon the application sending the data. The application requests the establishment of a session and prompts for the termination of established connected sessions for any standard, negotiated, termination.

In the case of the attack against Smaller Company the reliable delivery mechanism of TCP/IP is leveraged.

As mentioned previously the establishment of a TCP session is commonly referred to as the Three Way Handshake. The following is an expanded description of the process for two hosts, the originator and the destination where the destination is actively listening for the originator's connection on a defined port.

The originating host sends a packet containing a synchronization request to the destination host. The synchronization packet is defined by the presence of the SYN flag and an initial sequence number.

The destination host's reply includes an acknowledgement of the previous packet; the ACK flag is set and the incremented value of the initial sequence number is referenced. This same reply packet also contains a request for the establishment of a session by the destination host; the SYN flag is set and a new sequence number is referenced. Both the acknowledgement and the request for a new session are transmitted together to the originator in a single packet.

The originating host then replies, acknowledging the synchronization request. A packet containing the ACK flag set and the incremented value of the destinations sequence number is sent. The session is now established.

Due to the chaotic nature of the Internet, packets may take differing paths between two hosts and may in fact arrive in a different order than they were sent. The sequence numbers identify where in the stream of data a packet belongs.

The hosts are now free to pass data between them. They acknowledge each previous packet from the opposite host with a packet containing the ACK flag set and the incremented value of the previous packet's sequence number along with their own sequence number. In the case of originator pushing data to the destination, the sequence number it acknowledges is the same value as acknowledged in the previous packet. Either host may initiate the transmission of data to the opposing host.

This transmission and acknowledgement exchange originates within the application layer, is passed across the Transport layer (using TCP) carried upon the network layer (IP) over the link layer. Within TCP/IP the packet placed upon the link layer begins with the IP header and ends with its data segment. The data segment of the IP header contains the TCP header and its data segment. The contents of the TCP data segment depend upon the application involved. If information is present, within the TCP data segment, which the destination host should pass to the application as soon as possible, the TCP push flag (PSH) is set. Multiple packets of data may be sent prior to a packet being sent with the PSH flag set.

A standard session termination negotiation begins with the terminating host sending a packet with the FIN flag set, indicating a finish request, and a new sequence referenced. Also in this packet the terminating host acknowledges the previous packet of the session. The ACK flag is set and the incremented sequence number of the previous packet sent by the destination host is referenced.

The host receiving the termination request acknowledges the finish request with a packet containing the ACK flag set and the incremented sequence number of the previous packet. The responding host then issues a finish request of its own. A new packet with the FIN flag set and the new sequence number is sent. Also in this same packet is an acknowledgement of the previous packet received, in this case the FIN request of the terminating host. The ACK flag is set and the same incremented sequence value, for the previous acknowledgement of the terminating host's finish request, is referenced.

The terminating host acknowledges the finish request and the connection is closed.

A more detailed description of TCP specifically can be found under RFC793 . I would also highly recommend TCP/IP Illustrated Volume 1 by W. Richard Stevens.

How the Exploit Works:

Checkpoint Firewall-1 is a stateful inspection firewall. By stateful we mean that the firewall retains information about previously observed packets and the states of existing sessions for use in evaluating subsequent packets.

If the firewall receives a fragmented packet it waits until it receives all parts of that packet and then reassembles and evaluates the completed packet.

Upon receipt of a complete packet the Checkpoint Firewall-1 compares the packet to its listing of established sessions in the connection table. If the new packet is part of an existing session the packet is then forwarded. The examination against the list of established sessions is performed instead of a packet-by-packet comparison against the policy as a matter of efficiency. If new packets modify the state of an existing connection, the state change is noted and the packet forwarded.

That last statement primarily applies to the shutting down of a connection via FIN negotiations or abrupt terminations by way of reset (RST) packets. After the firewall has seen two FIN packets associated to a session the connection table entry is removed. It takes between 20 and 50 seconds for the process of updating the connection table to complete.

Connections are also removed if they exceed an idle timeout setting. For TCP sessions the default is 3600 seconds. UDP is a connectionless protocol. Checkpoint Firewall-1 emulates ¡§connectedness¡¨ for UDP sessions by tracking session initiation and holding open a return channel until a timeout is exceeded. For UDP ¡§virtual¡¨ sessions the timeout is 120 seconds. These values are set within the Policy Properties under the Security Policy tab.

The entire connection table is cleared if the firewall policy is reloaded. In environments where Checkpoint Firewall-1's session re-validation is not enabled, implementing policy changes terminates all active session on the affected firewall. This is an important consideration in many (if not all) production environments.

When the Firewall receives a SYN packet for the initiation of a new connection the packet is evaluated against the firewall policy. If the packet is initiating a new permitted session, a new entry is created in the connection table for that session. If the policy does not permit a session of the type defined by the inbound packet, either a reset packet is sent in response or the packet is silently dropped. The reset packet is constructed with the RST flag set and an acknowledgement of the previous packet (ACK flag set and the incremented value of the previous sequence number). Resets packets are implemented when it is desirable to notify the connecting host that the port in question is not available. Packets are dropped silently when it is desirable to preserve the anonymity of the state of the ports on the destination host.

For all other packets arriving at the firewall (i.e. not session initiating nor session terminating, and not part of an established session) the firewall policy is examined. If a rule is present that would have permitted the initiation of a session of the type defined by the packet, had it been a synchronization request, a connection table entry is created. Or stated differently, if a packet arrives claiming to be part of an active connection (that is permitted) it is given a new entry in the connection table. That packet is however, not forwarded.

Checkpoint Firewall-1 instead mutilates the packet. The data is removed (to eliminate the risk of a dangerous payload) and the sequence number is mangled. The new mangled packet is forwarded. If the target host expects a connected session from the source of the packet it will respond with a retransmission request. If it is not expecting a connected session it will send a reset.

When the firewall receives a valid response, i.e. the request for a retransmission, it validates the session in the connection table, resets the idle timeout and forwards the pending packet. If the firewall receives a reset in response to the mutilated packet it drops the pending entry from the connection table and does not forward the packet or respond to the errant ACK packet.

What remains is the case where the host fails to respond at all. In this eventuality the pending connection in the connection table remains until it exceeds its idle timeout.

Provide an excess of sessions initiated with ACK packets and the connection table will fill. Provide enough sessions and new connections cannot be established. The result is a Denial of Service.

If the firewall received a packet in fragmented form initially it re-fragments it during the process of forwarding. Checkpoint Firewall-1 has issues with its management of IP fragments. Checkpoint refers to logging issues being the source of the problem. The symptom is a significant CPU utilization increase. A large volume of incomplete fragmented packets causes consumption of resources on the firewall host. Excessive volume leads to resource starvation and an effective denial of service results.

The command line required to produce the ACK DoS (Nmap V. 2.54BETA25) is:

nmap ¡VsA <ip address>/<subnet>

Where ¡VsA selects the ACK scan and you define the scan range and the IP address (or addresses as defined by the CIDR subnet) include the firewall or hosts that reside beyond it. For the ACK DoS directed against smaller company the command line would have been: nmap ¡VsA X.Y.Z.1/28

The command line required to produce the Fragment Driven DoS

(Nmap V. 2.54BETA25) is:

nmap ¡Vf <ip address>/<subnet>

Where ¡Vf selects tiny fragmented packets and the IP address (or addresses as defined by the CIDR subnet) include the firewall or hosts that reside beyond it. By not defining a scan type Nmap defaults to a connect scan. For the Fragment-Driven DoS directed against smaller company the command line would have been: namp ¡Vf X.Y.Z.1/28

See the Nmap manual page for currently arguments and more information:

The Network Affected:

Smaller Company's WAN connected their regional offices to their main offices. Internet access was provided to Smaller Company by their ISP division. In their main offices a Cisco 7200 Router directly connected their corporate firewall to their ISP backbone. It is important to point out that within both Larger Company and Smaller Company the ISP services division functions independently of the corporate services division. This includes independent Network Security functions. It is also important to point out that Smaller Company's ISP division managed a large cable modem network. The cable modem network was also directly connected to their ISP backbone at the main offices.

As part of integrating the two companies Larger Company had installed a circuit into Smaller Company's main office. We were not comfortable with the existing security posture at Smaller Company. To protect Larger Company, and provide an auditable record of activity, we terminated the circuit between the two networks into a firewall. This firewall was configured to only permit access to Larger Company's email and Back Office systems on their specific client ports. Larger Company itself had multiple firewalls connecting its WAN to the Internet.

For reasons of business continuity we were forced to accept a permissive and poorly defined firewall policy until we could complete an audit of existing systems and develop and accurate map of business network connections.

Smaller Company's corporate firewall provided external access to several servers in RFC1918 space behind the firewall. This was accomplished through inbound static network address translation (NAT) from public IP space.