Title: CHKRootKit : Eliminate The enemy within
Scope of article
Last month we learnt about the technical components that form the rootkits. We also discussed why and how the rootkits are dangerous and glanced at a few detection options. This article is an extension to the same topic, talking about a proven method to detect the presence of rootkits on the servers. In this article, we are going to learn about a powerful utility named Chkrootkit, which is an open source tool dedicated for this purpose. We are going to know the installation methods and its usage, and also going to discuss ways to incorporate it in the daily administrative technical tasks, to ensure the security of servers and datacenters.
Revisiting the basics
In past several years, the open source Linux distributions have emerged from being a mere desktop operating system, to a rock solid platform for the production data centers, which cater to mission critical IT infrastructure. This growth prompted hackers to try exploiting into the kernel and make it susceptible to the attacks. The most serious form of such bad intentions was the introduction of rootkits, which are found to be seriously dangerous than viruses and trojans.
Rootkits are a concise collection of tools, utilities and scripts which are installed on a computer by hackers in order to take its control and exploit the system further by making use of those tools. The control is meant to gain administrator-level access of the target machine, so that the system can be controlled either remotely or locally. The motive behind such a control is either to steal data or to use that machine as a zombie by converting it into an attacker for other machines in the network, or to spread and install viruses. Root kits are composed of network sniffers, log parsers and other low level scripts which are used to gain root access. The real challenge in detecting rootkits stems out of the fact that when a rootkit is active and has gained necessary control, it can be capable of fooling an antivirus by modifying its output on the fly, and being transparent to the user or system administrator, leaving the rootkit undetected.
Typical symptoms of the presence of a rootkit can range from degraded overall system performance; file deletions etc up to altered file permissions, hindrance to backups and other administrative chores etc. Server management should comprise of a powerful monitoring system which can keep a close watch on the infrastructure health and alert such changes to the system administrators duly. System administrators need to have a process and set of tools to be used, for swiftly responding to such alerts and occurrences. For all the symptoms mentioned above, it is advised to first suspect a rootkit attack on the server, primarily due to the destructive nature of it.
Chkrootkit is a collection of tools to detect presence of rootkits, and is a gift to Linux system administrators for two specific reasons. One being that, it is a free open source utility and available for multiple distros. Second reason is that, it scans and detects almost all latest rootkits out there, and also the open source contributors community keeps it up to date, in order to capture latest kit attacks. Over the period the quality of Chkrootkit scan engine has also improved making it faster, which is especially useful in performing detailed kernel checks against a number of supported kit detections.
Following are a few great features of Chkrootkit listed….
Detects more than 60 old as well as latest kits
Capable of detecting network interfaces which are in promiscuous mode
Can efficiently detect altered lastlog and wtmp files, to help detect intrusion
Easy command line access with least confusing and very direct options
Verbose output mode to help admins automate tasks
Chkrootkit uses C and shell scripts to perform a detailed process check and scans system binaries to detect kit signatures. It can either report these utilities upon detection, and in most cases it is capable of removing those too. Chkrootkit has few algorithms which can report trend of a possible rootkit even if it is not yet officially supported. Below is a list of programs that the Chkrootkit uses internally and explains in brief what each of it does.
Chkrootkit Program Purpose
Chkrootkit Main script to check for tampered system files
strings.c Detect and perform string replacement
ifpromisc.c Checks network interface for promiscuous mode
chklastlog.c, chkwtmp.c Checks if lastlog and wtmp entries are deleted
chkproc.c, chkdirs.c Check for Linux Kernel Module based Trojans
Chkrootkit installation is a straight forward process on Redhat and few other distributions. You can download the most recent tarball from www.chkrootkit.org/download to a temporary folder. It is recommended to perform md5 check and then decompress the tarball by using the command
tar xfvz chkrootkit.tar.gz
Once done, change to the directory where exploded files are copied by tar decompressor and run this command to compile it
For Debian Linux flavors such as Ubuntu, you can use the following command to download and install
sudo apt-get install Chkrootkit
The very first recommended step after installing Chkrootkit on a system is to run ifpromisc. This checks all the network interfaces for being in promiscuous mode, which should not be the case unless the system itself is infected with a rootkit, prior to the installation. Once this check is performed, the next step is to run the tool without any command line options. Please refer to Fig 1. that shows how a typical ubuntu system is scanned and checked for various supported anomalies and rootkits. Fig 2. shows how Chkrootkit checks all kernel processes and system files, based on an internally stored list of things to be checked. We can also see that if a particular operating system utility (such as fingerd seen in the figure) is not detected, it reports so accordingly. This helps administrative scripts to know if a file is deleted which normally should be present.
Though running Chkrootkit without any command line options is usually sufficient, the system administrators may want more flexibility to script it and use the tool in their daily work. The table below talks about few command options with an explanation and examples
Option Explanation Example
-h Shows help >chkrootkit –h
-v Shows version >chkrootkit -v
-l Lists tests supported >chkrootkit -l
[testname] Scan for a specific test Check Sniffer command for “ps” Trojan
>chkrootkit ps sniffer
-x Expert mode (displays each action taken on each file scanned) >chkrootkit –x | more
-q Quiet mode. Only displays if a binary is found to be “Infected” >chkrootkit –q
-r dir Use specified dir as root dir. Useful in scanning a suspicious machine from a healthy one. Also useful in scanning mounted volumes. Scan a volume mounted under “mnt1”
>chkrootkit –r /mnt1
It is also important to understand how Chkrootkit displays the output. Typically the phrases such as “not found”, “not infected” are displayed. When a rootkit is found, or if a possible rootkit is suspected to be present, the output highlights it with “INFECTED”, or “the following suspicious files or directories found”. Please refer to Fig. 3 which shows that a python script is being reported as a suspicion. These outputs can be captured to a text file log, or parsed with a grep command to remove the clutter and only concentrate on the infected areas. Using –q is another option available, however please note that it can also suppress possible suspicions which may not be a good idea.
What if a rootkit is found?
Chkrootkit does the job of detecting rookits, but does not remove those. Upon finding a rootkit on a system, the first thing to do is to disable that system from rest of the network to avoid spreading. To remove a rootkit in a cleanest way, we need to backup and rebuild the entire system, which however may not always be possible. Another approach is to study the detected rootkit thoroughly, and perform actions to remove it based on its way of intrusion and working. Many rootkits can be removed manually; however there are a few which need only the cleanest approach. To improve the detection accuracy, it is advised to run Chkrootkit from a known healthy system, against all servers in the farm.
Integrating Chkrootkit for admin tasks
System administrators are strongly advised to use Chkrootkit in their daily administrative tasks. By using an appropriate command line option, we can create a script to have verbose output and dump it into a log file which can be further parsed to look for anomalies. This script can be automated using a cron job to be run on a daily schedule. Over the period, the script can be tuned further to remove false alarms generated by suspicious files and report only the real problems. Another cron job can be scheduled for a frequent interval, to have Chkrootkit detect if the network interfaces are running in promiscuous mode or not. This is essential because usually a rootkit attack starts by tampering into network reconfigurations. Further scripting can be done to create a list of servers in the farm to be scanned and consolidate their outputs and alert or report administrators accordingly.
Rootkits are a serious threat to the modern datacenters. It is essential for IT management to have a definite means to detect it and take necessary preventive actions. Chkrootkit comes to the rescue for this task, being a fast and effective scanning tool just for this purpose. Administrators can learn various command line options available with this tool, and create automated tasks to make their IT infrastructure secure and stable.
About the author
The author has over 18 years of experience in the field of IT hardware, networking, web technologies and IT security. Prashant is MCSE, MCDBA certified and also F5 load balancer expert. In the IT security world he is an ethical hacker and net-forensic specialist.
Prashant runs his own firm named Valency Networks in India (www.valencynetworks.com) providing consultancy in IT security design, security audit, infrastructure technology and business process management. He can be reached at firstname.lastname@example.org.