Title

A Behavior Based Approach to Virus Detection

Document Type

Dissertation

Degree

Doctor of Philosophy

Department

Computer Science

Advisor's Name

Dr. Yi Deng

Advisor's Title

Committee Chair

Advisor's Name

Dr. Peter J. Clarke

Advisor's Title

Committee Co-Chair

Advisor's Name

Dr. G.M. Golam Kibria

Advisor's Title

Committee Member

Advisor's Name

Dr. Geoffrey Smith

Advisor's Title

Committee Member

Advisor's Name

Dr. Xudong He

Advisor's Title

Committee Member

Keywords

computer virus behavior based self reference repli

Date of Defense

3-24-2008

Abstract

Fast spreading unknown viruses have caused major damage on computer systems upon their initial release. Current detection methods have lacked capabilities to detect unknown virus quickly enough to avoid mass spreading and damage. This dissertation has presented a behavior based approach to detecting known and unknown viruses based on their attempt to replicate. Replication is the qualifying fundamental characteristic of a virus and is consistently present in all viruses making this approach applicable to viruses belonging to many classes and executing under several conditions. A form of replication called self-reference replication, (SR-replication), has been formalized as one main type of replication which specifically replicates by modifying or creating other files on a system to include the virus itself. This replication type was used to detect viruses attempting replication by referencing themselves which is a necessary step to successfully replicate files. The approach does not require a priori knowledge about known viruses. Detection was accomplished at runtime by monitoring currently executing processes attempting to replicate. Two implementation

prototypes of the detection approach called SRRAT were created and tested on the Microsoft Windows operating systems focusing on the tracking of user mode Win32 API system calls and Kernel mode system services. The research results showed SR-replication capable of distinguishing between file infecting viruses and benign processes with little or no false positives and false negatives.