Taming the Shrew: Modifying TCP to Withstand Denial of Service Attacks Florin Dinu Zheng Cai Rice University TCP is a cornerstone of today’s Internet. It is pervasive. Studies [1] have shown that TCP accounts for 80% of today’s Internet traffic. Because of its widespread use a lot of research has gone into tuning TCP’s parameters to maximize performance. There have, however, always been security issues surrounding the use of TCP. A comprehensive list was published in [2]. Other problems arise from TCP’s assumption that end-points should cooperate and use the same algorithms for congestion control when packet loss is encountered in the net- work. Yet another class of attacks has gotten a lot of attention recently. These attacks allow the throttling of TCP fiows to a small fraction of their ideal speed (Reduction of Quality attacks like [3]) and go as far as terminating TCP fiows by injecting reset packets (Denial of Service attacks). In this project we look at ways to modify TCP to make it immune to this last class of attacks. Reduction of Quality (RoQ) attacks take advantage of TCP’s back-off and retransmission mechanism to constantly create traffic loss and therefore make attacked fiows enter the slow-start phase very often. They use very short but high-rate bursts of traffic to create congestion at the same time that the attacked fiow tries to re-send lost packets. Such attacks are stealthy because they have a small average rate and therefore bypass detection systems installed in routers. There have been two main classes of solutions to alleviate this problem. One possible solution can be employed at router level and requires the identification of malicious fiows using patterns that can differentiate them from regular fiows [4]. The other approach modifies the TCP protocol itself. One straight-forward approach is the randomization of the back-off timer. This solution, however, presents an inherent trade-off between the performance of TCP and the resulting security gains. Modifications of the back-off algorithm lead to a decrease in TCP performance. In this project we look at other possible modifications that can create TCP-like transport protocols resistant to RoQ attacks. One major issue with TCP is that any modification must be backward compatible with current implementations. This inherently limits the amount of adjustment that can be done on the protocol and generally takes precedence in practice. In this project, however, we put emphasis on coming up with solutions for the attacks presented even if this necessitates changes in the protocol. We do try to keep the changes within reasonable limits. A first idea is to see how Explicit Congestion Notification (ECN) mechanisms can be leveraged to provide more information to the transport protocol about the level and type of congestion at the routers. This might give valuable information to TCP clients to enable them to differentiate congestion caused by RoQ attacks. Time permitting, we will also look at DoS attacks caused by injecting RST

packets. With enough information about the victim’s active connections and about the TCP sessions (sequence numbers) the attacker can send RST packets and disrupt the connections. To defend against this kind of attack, one idea is that in TCP, identification should not only depend on source/destination addresses and port numbers be- cause they can easily be spoofed. We still want to the functionality provided by the RST packets, but we want to make sure that a fake RST packet will not do any damage. There could either be a negotiation before the connection actually gets closed, or a fingerprint can be added to the RST packet so that only one of the peers can generate valid RST packets. To evaluate our approach, we first need to show that that it works in different attack scenarios while preserving the correct functionality of the original TCP. We also plan to evaluate how much overhead our approach adds compared to TCP, both in terms of packet size and communication time. We plan to implement our ideas using the ns2 [5] simulator.

References [1] K.-C. Lan, A. Hussain, and D. Dutta, “The effect of malicious traffic on the network,” in PAM 2003, La Jolla, April 2003. [2] S. M. Bellovin, “Security problems in the tcp/ip security suite,” in Computer Communication Review, April 1989. [3] Kuzmanovic and E. W. Knightly, “Low-rate tcp-targeted denial of service attacks (the shrew vs. the mice and elephants),” in ACM SIGCOMM, August 2003. [4] Y. Chen and K. Hwang, “Tcp flow analysis for defense against shrew ddos attacks,” in ICC, 2007. [5] “The ns2 Network Simulator.” http://nsnam.isi.edu/nsnam/index.php/ Main Page.

2