Host based Attack Detection using System Calls Jestin Joy (141103) Under the Guidance of Ms Anita John Rajagiri School of Engineering & Technology

April 3, 2012

Jestin Joy (RASET)

Attack Detection

April 3, 2012

1 / 24

Outline 1

Motivation Approaches to Intrusion Detection

Jestin Joy (RASET)

Attack Detection

April 3, 2012

2 / 24

Outline 1

Motivation Approaches to Intrusion Detection

2

System Call

Jestin Joy (RASET)

Attack Detection

April 3, 2012

2 / 24

Outline 1

Motivation Approaches to Intrusion Detection

2

System Call

3

Current Work Kernel Level Rootkits Attacks Host Based Anomaly Detection Our Approach Results & Discussion

Jestin Joy (RASET)

Attack Detection

April 3, 2012

2 / 24

Outline 1

Motivation Approaches to Intrusion Detection

2

System Call

3

Current Work Kernel Level Rootkits Attacks Host Based Anomaly Detection Our Approach Results & Discussion

4

Future Work

Jestin Joy (RASET)

Attack Detection

April 3, 2012

2 / 24

Motivation

Motivation Motivation As systems are connected to internet, they became more vulnerable to attack. These attacks can be handled at both network and host level. Some attacks can be from sources other than network. This calls for better host based intrusion detection[1]. Intrusion Detection Intrusion detection is the process of monitoring the events occurring in a computer system or network and analysing them for signs of possible incident.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

3 / 24

Motivation

Approaches to Intrusion Detection

There are two basic approaches: 1

Misuse Detection : A system using this approach detects intrusion events which follow well-known patterns.

2

Anomaly Detection : A system using this approach looks for deviation from normal behaviour. A system call is a request for service that a program makes to the kernel. The service is generally something that only the kernel has the privilege to do, such as doing I/O. Attackers make use of system calls for initiating the attacks

Jestin Joy (RASET)

Attack Detection

April 3, 2012

4 / 24

System Call

System Call in Linux

Jestin Joy (RASET)

Attack Detection

April 3, 2012

5 / 24

Current Work

Initial work focussed on rootkits. Rootkits Rootkits[2] are a set of software tools used by an attacker to gain unauthorized access into a system, thereby providing him with privilege to access sensitive data, conceal its own existence and allowing him to install other malicious software. Rootkits are the most challenging malware to detect due to their elusive nature. Modern rootkit attacks mainly focus on modifying operating system kernel.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

6 / 24

Current Work

Kernel Level Rootkits

Kernel Level Rootkits

Kernel level rootkit[3] modifies the kernel itself. Kernel being the lowest level of operating system makes it a good choice for the intruder to attack, since an attack on it is very difficult to detect. Also being at the kernel level provides the attacker with complete freedom to access all most all areas of an operating system.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

7 / 24

Current Work

Attacks

Attacks

Figure: System Call Table Modification Jestin Joy (RASET)

Attack Detection

April 3, 2012

8 / 24

Current Work

Attacks

Attacks

System Call Table Modification : Attacker replaces original system call with his own custom version System Call Target modification: Legitimate code in the target address is modified System Call Table Redirection: Attacker replaces the call to System Call Table with his own custom version.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

9 / 24

Current Work

Host Based Anomaly Detection

Detection

Due to its efficiency and easiness attackers increasingly use system calls for attack Current techniques for detection mainly rely on archived copy of system files for detection[4] Need a detection approach that takes less prior information about the state of the system

Jestin Joy (RASET)

Attack Detection

April 3, 2012

10 / 24

Current Work

Our Approach

Our Approach

The proposed technique detects system call redirection attack which is a widely employed kernel level rootkit attack[5]. Our approach doesn’t need prior information about the state of the system to function[5] Detection mechanism uses statistical information from the infected system System call redirection attacks cause anomaly to the host system. Creation of clusters[6] could help to detect the attacks

Jestin Joy (RASET)

Attack Detection

April 3, 2012

11 / 24

Current Work

Our Approach

Our Approach Main Idea When a LKM based attack using new system call occurs, it gets an unusually high memory address as compared to other system calls. For the detection purpose the running kernel should be disassembled. We scan the system call table and the unconditional/conditional jump instructions are analysed and their operands, which are memory addresses are collected. Attackers make use of higher level system calls which are loaded earlier in memory for manipulation. We do clustering to differentiate between normal and malicious system calls

Jestin Joy (RASET)

Attack Detection

April 3, 2012

12 / 24

Current Work

Our Approach

Our Approach

Figure: Data flow diagram

Jestin Joy (RASET)

Attack Detection

April 3, 2012

13 / 24

Current Work

Our Approach

Our Approach

Figure: Data flow diagram

Jestin Joy (RASET)

Attack Detection

April 3, 2012

14 / 24

Current Work

Our Approach

Results & Discussion 2500000 +3.238e9

System Call Distribution

Memory Addresses

2000000 1500000 1000000 500000 00

50

100

150 200 System Calls

250

300

350

Figure: System Call Distribution

Figure represents system call distribution in an uninfected system. (kernel 2.6.38 ) Jestin Joy (RASET)

Attack Detection

April 3, 2012

15 / 24

Current Work

Our Approach

Results & Discussion No 1 2 3 4 5 6 7 8 9 10 11 12

System Call No 102 253 277 278 279 280 281 282 286 287 288 337

System call SOCKETCALL LOOKUP DCOOKIE MQ OPEN MQ UNLNK MQ TIMEDSEND MQ TIMEDRECEIVE MQ NOTIFY MQ GETSETATTR ADD KEY REQUEST KEY KEYCTL RECVMMSG

Address c11f6c23 c1111456 c1119279 c1119117 c1118eac c1118af3 c11186e2 c11185ac c111be4e c111bfb0 c111c22b c11f58aa

Figure shows system calls that fell into false cluster Jestin Joy (RASET)

Attack Detection

April 3, 2012

16 / 24

Current Work

Our Approach

Results & Discussion System Call Distribution

4.2 1e9

Memory Addresses

4.0 3.8 3.6 3.4 3.20

50

100

150 200 System Calls

250

300

350

Figure: System Call Distribution

Figure represents system call distribution in an infected system. (kernel 2.6.38 ) Jestin Joy (RASET)

Attack Detection

April 3, 2012

17 / 24

Current Work

Our Approach

Results & Discussion

False positive rate is 0.0366% Analysis could be done on a machine other than the host system. The presence of natural outliers is a bottleneck In-order to detect other type of system call based methods like, system call target modification, the contents of the system call table need to be disassembled. Disassembling each system call gives us information about different points in the call that are candidates of hooking.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

18 / 24

Current Work

Our Approach

Results & Discussion GDB can be utilized to disassemble our running kernel. # gdb / u s r / s r c / l i n u x − 2 . 6 . 3 8 . 8 / v m l i n u x / p r o c / k c o r e ( gdb ) d i s a s s s y s w r i t e Dump o f a s s e m b l e r code f o r f u n c t i o n s y s w r i t e : 0 xc10d2c83 < s y s w r i t e +0>: push %ebp 0 xc10d2c84 < s y s w r i t e +1>: mov %esp ,% ebp 0 xc10d2c86 < s y s w r i t e +3>: push %e s i 0 xc10d2c87 < s y s w r i t e +4>: mov $ 0 x f f f f f f f 7 ,% e s i 0 x c 1 0 d 2 c 8 c < s y s w r i t e +9>: push %ebx 0 xc10d2c8d < s y s w r i t e +10>: s ub $0x10 ,% e s p 0 xc10d2c90 < s y s w r i t e +13>: mov 0 x8(%ebp ) ,% ea x 0 xc10d2c93 < s y s w r i t e +16>: l e a −0xc(%ebp ) ,% edx 0 xc10d2c96 < s y s w r i t e +19>: c a l l 0 xc10d34bd < f g e t l i g h t 0 xc10d2c9b < s y s w r i t e +24>: t e s t %eax ,% ea x Jestin Joy (RASET)

Attack Detection

April 3, 2012

19 / 24

Current Work

Our Approach

Results & Discussion

The vmlinux file is the uncompressed kernel image The optional /proc/kcore parameter acts as a core file, to let gdb actually peek into [7] the memory of the running kernel. There are jump instructions, which can be abused by an attacker to divert the normal control flow. Attack simulation is a challenge

Jestin Joy (RASET)

Attack Detection

April 3, 2012

20 / 24

Future Work

Future Work

Detection daemon is invoked every 30 seconds, which induces an extra load in the system. Embedded devices are also victims of these types of attacks. Study should be done on them. The effect of detection mechanism on low memory devices should be thoroughly studied. Our prototype has included only system call addresses for the detection purpose. System call sequences could also be studied [8]

Jestin Joy (RASET)

Attack Detection

April 3, 2012

21 / 24

Future Work

References I

S. Freeman, A. Bivens, J. Branch, and B. Szymanski, “Host-based intrusion detection using user signatures,” in Proceedings of the Research Conference. RPI, Troy, NY, 2002. J. G. Levine, J. B. Grizzard, and H. L. Owen, “Detecting and categorizing kernel-level rootkits to aid future detection,” IEEE Security and Privacy, vol. 4, pp. 24–, January 2006. J. Levine, J. Grizzard, and H. Owen, “A methodology to detect and characterize kernel level rootkit exploits involving redirection of the system call table,” in Information Assurance Workshop, 2004. Proceedings. Second IEEE International, pp. 107–125, IEEE, 2005.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

22 / 24

Future Work

References II G. H. Kim and E. H. Spafford, “The design and implementation of tripwire: a file system integrity checker,” in Proceedings of the 2nd ACM Conference on Computer and communications security, CCS ’94, (New York, NY, USA), pp. 18–29, ACM, 1994. J. Joy, A. John, and J. Joy, “Rootkit detection mechanism: A survey,” Advances in Parallel Distributed Computing, pp. 366–374, 2011. J. Joy and A. John, “A host based kernel level rootkit detection mechanism using clustering technique,” in Trends in Computer Science, Engineering and Information Technology (D. Nagamalai, E. Renault, and M. Dhanuskodi, eds.), vol. 204 of Communications in Computer and Information Science, pp. 564–570, Springer Berlin Heidelberg, 2011.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

23 / 24

Future Work

References III

R. Stallman, R. Pesch, S. Shebs, et al., Debugging with GDB. Gnu Press, 2002. S. Forrest, S. Hofmeyr, A. Somayaji, and T. Longstaff, “A sense of self for unix processes,” in Security and Privacy, 1996. Proceedings., 1996 IEEE Symposium on, pp. 120 –128, may 1996.

Jestin Joy (RASET)

Attack Detection

April 3, 2012

24 / 24

Host based Attack Detection using System Calls

Apr 3, 2012 - This calls for better host based intrusion detection[1]. ... Intrusion detection is the process of monitoring the events occurring in a ... System Call in Linux ... Rootkits[2] are a set of software tools used by an attacker to gain.

407KB Sizes 4 Downloads 313 Views

Recommend Documents

Detection of Malicious Web Pages Using System Calls Sequences ...
In this paper we propose to use system calls to detect malicious JavaScript. ... In: Proceedings of the 2nd USENIX Conference on Web Application Development, p. 11. ... Roy, C.K., Cordy, J.R.: A survey on software clone detection research.

Host Based Intrusion Detection and Countermeasure Selection in Cloud
Particularly, intruders can exploit vulnerability to a cloud system and compromise virtual machines to deploy further large scale types of attack like distributed ...

Rogue Access Point Detection and Counter Attack Using Internet Proxy
www.ijrit.com. ISSN 2001-5569. Rogue Access Point Detection and Counter Attack. Using Internet Proxy. Miss. Gaikwad Jyoti, Miss. Mandhare Ashvini, Miss.

Rogue Access Point Detection and Counter Attack Using Internet Proxy
3. Host policies and rules are stored in the MYSQL database on proxy server. 4. Proxy will check the host policy and process the request accordingly. 5. User gets the internet access if he is an authorized user otherwise gets the error message. 6. Pr

2009_TRR_Draft_Video-Based Vehicle Detection and Tracking Using ...
2009_TRR_Draft_Video-Based Vehicle Detection and Tracking Using Spatiotemporal Maps.pdf. 2009_TRR_Draft_Video-Based Vehicle Detection and Tracking ...

Using Trustworthy Host-Based Information in the Network
18 Nov 2009 - Once authorized by a verifier, the code module can insert cryptographically- secured information into outbound traffic. This information is checked and ...... ±990 (+0.2%). 6631. ±721 ( +12.8%). Up L. 706.5 ±61.4. 777.4 ±153 (+10.0%

Using Trustworthy Host-Based Information in the Network
Nov 18, 2009 - Once authorized by a verifier, the code module can insert cryptographically- secured information into outbound traffic. This information is checked and acted on by in-path filters. ... veying host-based information to the network, we i

Detection of Mysticete Calls
Abstract. This paper presents a methodology for automatically detecting mysticete calls. This methodology relies on sparse representations of these calls combined with a detection metric that explicitly takes into account the possible presence of int

Profile Injection Attack Detection for Securing ... - CiteSeerX
6.4.2 Robustness Comparison Against Nuke Attacks . . . . . . . . . . . . . . 34. 7 Defense ..... As a result, we have focused on profile analytics data and attribute ...... Data Mining: Practical machine learning tools and techniques, 2nd. Edition. M

Profile Injection Attack Detection for Securing ... - CiteSeerX
to Bamshad Mobasher for inspiring and encouraging me to pursue an academic career in computer science. His thoroughness and promptness in reviewing my ...

Design and Implement Online SQL Injection Detection System Based ...
There was a problem previewing this document. ... Design and Implement Online SQL Injection Detection System Based on Query Signature.pdf. Design and ...

Enhanced Group Signature Based Intruder Detection System ... - IJRIT
Keywords- Digital signature, digital signature algorithm (DSA), Enhanced Group Signature Based Intruder Detection System (EGIDS), Mobile. Ad hoc NETwork ...

Enhanced TCP SYN Attack Detection
prevalent in the Internet, with attacks targeting banking and financial companies, online gambling firms, web retailers and governments. The 2007 Symantec Threat Report [2] indicates that over 5000 DoS attacks were observed worldwide on a daily basis

Sparse Representation based Anomaly Detection using ...
HOMVs in given training videos. Abnormality is ... Computer vision, at current stage, provides many elegant .... Global Dictionary Training and b) Modeling usual behavior ..... for sparse coding,” in Proceedings of the 26th Annual International.

Automated Detection of Engagement using Video-Based Estimation of ...
Abstract—We explored how computer vision techniques can be used to detect ... supervised learning for detection of concurrent and retrospective self-reported engagement. ...... [49] P. Ekman and W. V. Friesen, Facial Action Coding System: A ... [On

Fault Detection Using an LSTM-based Predictive Data ...
data set with labeled faults, we used an LSTM architecture with a forecasting error threshold to ... data. Numerous approaches to fault detection (FD) in industrial and other types of multivariate time series have been proposed: classic methods like

Enhanced Group Signature Based Intruder Detection System ... - IJRIT
IJRIT International Journal of Research in Information Technology, Volume 2, Issue 4, ... (MANET) is a collection of mobile nodes equipped with both a wireless.

signature based intrusion detection system pdf
signature based intrusion detection system pdf. signature based intrusion detection system pdf. Open. Extract. Open with. Sign In. Main menu. Displaying ...

Automated Detection of Engagement using Video-Based Estimation of ...
Abstract—We explored how computer vision techniques can be used to detect engagement while ... supervised learning for detection of concurrent and retrospective self-reported engagement. ...... [Online]. Available: http://msdn.microsoft.com/en-us/l

Feature Selection for Intrusion Detection System using ...
Key words: Security, Intrusion Detection System (IDS), Data mining, Euclidean distance, Machine Learning, Support ... As the growing research on data mining techniques has increased, feature selection has been used as an ..... [4] L. Han, "Using a Dy

Detection of Malicious Web Pages Using System ... - Gerardo Canfora
Existing techniques for detecting malicious JavaScript suffer from ..... SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.

Detection of Malicious Web Pages Using System ... - Gerardo Canfora
Existing techniques for detecting malicious JavaScript suffer from some limitations like: the ... A system for circumventing these hurdles is to analyze the web threats at a finer grain, which is the one ..... Development, p. 11. USENIX Association .

Leak detection system
Sep 12, 1988 - a series of precisely-aligned and formed spiral grooves that cooperate with several parallel helical grooves formed in the stator. The use of the ...