IR and forensic talk

In this second post in my short series of timeline analysis I’m going to discuss the use of the CSV output module.  In my previous post I discussed a bit about the different modules there are in log2timeline, at least the version that was released then, and the meaning of each entry within the mactime format.  However since then, I’ve started to use the CSV output module instead of the mactime one, which will be the focus today.

After reading the excellent blog post about reviewing timelines using Excel by Corey few months ago I immediately told my self…that’s exactly what I was planning to do for my second series of the timeline analysis post.  However, I wanted to focus more on the CSV output and the benefits of using that, so I decided to a similar blog post as he did, just with a alternative method.  Somehow this blog post just got buried away somewhere far, far away and never got written… well, until now.  You can read the original post by Corey here, and I’m not going to try to repeat what is said there, please read the post yourselves as it contains lots of useful information that I’m not including here.

The problem I’ve got with the mactime format and Excel is the fact that filtering is not really optimal when it comes to something like the supertimeline.  That is to say, it is easy to filter out dates, but as soon as you start trying to filter the description field you start to hit the limitations of Excel quite quickly, just like Corey pointed out.  However, the CSV output module of log2timeline splits up the information into more fields, allowing greater and more fine grained filtering, making the basic filters more useful and the need to go to advanced filters less likely.  So I’m going to focus on that aspect in this post.  The CSV output consists of the following fields:

date,time,timezone,MACB,source,sourcetype,type,user,host,short,desc,\
version,filename,inode,notes,format,extra

The fields meaning is:

• Date: The date of the event, in the format of MM/DD/YYYY
• Time: The time of day, expressed in a 24h format, HH:MM:SS
• Timezone: the timezone that was used to call the tool with.
• MACB: The MACB meaning of the fields, mostly for compatibility with the mactime format.
• source: The short name for the source.  All web browser history is for instance WEBHIST, registry entries are REG, simple log files are LOG, etc.
• sourcetype: A slightly more comprehensive description of the source, “Internet Explorer” instead of WEBHIST, “NTUSER.DAT” instead of REG, etc.
• type: The type of the timestamp itself, such as “Last Accessed”, “Last Written” or “Last modified”, etc.
• user: The username associated with the entry, if one is available.
• host: The hostname associated with the entry, if one is available.
• short: A short description of the entry, usually contains less text than the full description field.
• desc: The description field, this is where most of the information is stored, the actual parsed description of the entry.
• version: The version number of the timestamp object.
• filename: The filename with the full path of the filename that contained the entry
• inode: The inode number of the file being parsed.
• notes: Some input modules insert additional information in the form of a note, which comes here.  Or it can be used during the review by the investigator.
• format: The name of the input module that was used to parse the file.
• extra: Some additional information parsed is joined together and put here.

Now we only need to create the timeline… the first step is to generate a timeline using TSK, save to a file, let’s call it “bodyfile”. Then to generate the timeline using log2timeline (in this case we are dealing with a Windows XP image):

cd /mnt/analyze
timescanner -d .  -z local -f winxp -o csv -w /cases/123456/timeline.txt

Now you got two files, one being in CSV format the other in mactime (the one produced by TSK).  Now we need to convert the mactime format into CSV, again using log2timeline to do so. We use the mactime input module to read in the bodyfile, and we append it to the timeline.txt file we created earlier.

log2timeline -f mactime -z local -o csv -w timeline.txt bodyfile

This way we have a file, “timeline.txt”, which is a CSV file containing both the timeline extracted using TSK and log2timeline.  We can now open this file up in Excel.  I’m using the Mac OS X version, so the screenshots might differ a bit from the Windows version, yet the principles should all stay the same (Windows screenshots can be found in Corey’s post here).

Import the File Into Excel

The first step obviously is to open Excel and import the file.  That can be done in two ways, either simply opening the file itself or by choosing “File/Import”.

If you decided to go for the “import” function, the next screen is a choice of file type.  Here we will choose a text file instead of the default value of a CSV file.

Choosing the file type in the import menu

The next step of the text import wizard gives us the choice of either splitting the file using a fixed with or delimited.  Since this is a comma separated file, we will choose “Delimited”.

The file is comma delimited, so choose delimited.

The third step of the wizard let’s us choose which delimiters we’ve got.  We only want to use the “comma” option, so please un-check the “Tab” field and check the “comma” one.

Check the box marked "comma", and un-check the box "tab"

The third and final step of the wizard is to choose the data type for each column.  I usually choose the value of the “date” field as a “Date” of the format “MDY” to make things a bit easier in the final step.

In the last window in the text import wizard, choose the date field as the value "Date: MDY"

Sorting

Now all the data is imported, one thing to notice is the fact that timescanner does not sort the timestamps at all.  This is due to the fact that timescanner recursively scans the image, and parses each file it finds, inserting the events as it goes. Therefore we need to sort the data, and create filters.  Start by highlighting the top row and choose “Filter” from the “Data” menu.

Highlight fields and choose "Data/Filter"

This turns all of our fields inside the top row into filterable columns, with a drop-down menu.  Now all we need to do is to sort the data in the correct time order.  So we go for the “Data / Sort…” option in the menu.

Time to sort the data

To correctly sort the data we start by sorting the fields based on the “Date” column, and we usually want to have the latest events on the top, so we choose to sort on value in the order of “Newest to oldest”.  However this is not enough, since we’ve also got the time field.  We therefore add another field using the “+” sign, there we choose the column “time”, which we also sort on values based on the order of “Largest to Smallest”.

Sorting the timestamp based on date from newest to oldest and time on largest to smallest

Now we’ve got our timeline all sorted out and ready to analyse.

Analysis

Just to show the simple filters and what we can do with them.  Now we can easily sort based on the sourcetype for instance.  Just click on the arrow next to the “sourcetype” column and choose which sourcetypes you would like to include in the analysis.

Filtering based on sourcetype

Now you can easily choose which days to examine, months or years.  Simply use the “date” column and filter based on that.  Here we are only focusing on January 2009 (or janúar as it is written in Icelandic).

Filtering dates, simple filter

One final trick I use quite frequently with the simple filters.  As soon as I’ve gone through some timestamps that are of some value, I often highlight them and give them color, usually only use few colors.  Then I can filter out or include only events of certain color.

Fields can be color coded

Filtering based on color:

Finally the CSV output contains often too many columns.  So it is often wise to hide few columns from view.  I usually hide the following fields:

• timezone (the same for everyone usually)
• host (if I’m examining one host only on the timeline)
• short (better to see the full description)
• version (don’t care about that one)
• format

One thing to notice though is the fact that timelines can quickly become way too large for Excel and other spreadsheet application.  The spreadsheet becomes incredibly slow and difficult to manage.  So I usually use some grep and other command-line kung-fu to remove irrelevant entries from the timeline, or only include the timespan of the investigation, that is making it as small as possible.

In my next post, I will go over  some of the command line stuff I usually do to minimize my timeline and other tricks I tend to do before loading it up in Excel.

I will also go over visualization, using the output module of BeeDocs.

And if you have any other suggestions, please add them in the comment section.

Timeline analysis has been getting a lot of press lately.  Harlan discussed some of the sources and usability of timeline analysis in a recent blog post. And then you’ve got few posts that describe how to create timelines, both from a live Windows machine, and from registry files. Rob Lee also posted a blog about creating timelines from shadow volume copies.

Then log2timeline has been getting some discussion as well. Paul Bobby actually pointed some bugs to me as well as posting two posts on his blog, one being a discussion of  the issues of mounting the image file using Encase and accessing log2timeline from a virtual machine. The other one about an Encase script to extract all the files that log2timeline is capable of parsing to a directory (at least most of them), to make it easy for log2timeline to parse it without the need to mount the image file.

Rob Lee also posted a blog post about super timeline creation, using among other tools, log2timeline.  I will post similar posts as Rob soon, just have to complete the new version of the tool first.  The plan is to complete it, which I hope will add some good improvements, before the SANS EU forensics summit. And speaking of the summit, the detailed agenda for the SANS EU forensics summit is up, make sure you don’t miss it.

Update 1

Timeline analysis can be extremely useful during any investigation.  Although traditional file system timeline can be very helpful it sometimes misses important events that are stored inside files.  These events might be crucial to the investigation or at least provide a better view of the events that really occurred on the suspect system. So to get the big picture, or a complete and accurate description we need to dig deeper and incorporate information found inside artifacts or log files into our timeline analysis. These artifacts or log files could reside on the suspect system itself or in another device, such as a firewall or a proxy (or any other device that logs down information that might be relevant to the investigation).

Unfortunately there are few tools out there that can parse and produce body files from the various artifacts found on different operating systems to include with the traditional filesystem analysis. Harlan Carvey has been working on some scripts for the Windows platform to accomplish this, such as regtime.pl to create a body file from the registry. Usually these tools are build specifically to parse a file/artifact that is of a particular format (such as a tool just to produce a body file from restore points). I’ve released some tools like that, as well as H. Carvey and others. I know of one attempt to create a framework to correlate different artifacts into a timeline, a project called Ex-Tip, by Mike Cloppert. There is a GCFA gold paper describing the framework. This project was started in May 2008, but hasn’t been maintained since then. Instead of extending that project I decided to start my own, that is to add a tool that can correlate information found inside different log files and artifacts into the traditional timeline analysis. I wanted to be able to easily integrate this tool into already existing tools that deal with timeline analysis, so I chose to output all timelines in a mactime body format, to be used with the tool mactime from TSK (The SleuthKit). This tool is called log2timeline and already supports incorporating seven different artifacts into the timeline.

In other words, this tool has been created to use artifacts and log files found on suspect systems (and others) in a timeline analysis to assist the investigator so that he can more easily see the “big picture”. That is to be able to build a more accurate timeline of the events that have occurred and when (and in which order).  Such a tool has to have a wide range of support for different log files and artifacts to be useful for investigators, yet despite only being capable of parsing six artifacts today I would like to publish my first beta version of the tool for people to download and try out.  Current version of the tool parses the following artifacts:

• Prefetch directory (reads the content of the directory and parses files found inside)
• UserAssist key info (reads the NTUSER.DAT user registry file to parse the content of UserAssist keys)
• Squid access logs (with emulate_httpd_log off)
• Restore points (reads the content of the directory and parses rp.log file inside each restore point)
• Windows shortcut files (LNK)
• Firefox history (for version 3.+)
• Windows Recycle Bin (INFO2)

Although not nearly enough support for different artifacts, at least it is a start.  Future versions will support at least:

• Event Logs
• Index.dat files (IE History)
• FF files (FF History older version)
• ISA text export
• Squid access log with httpd_emulate equal to on
• Cisco ACL entries
• Linux syslog
• pcap dump files
• Mac OS X artifacts
• Other Linux artifacts
• Opera and Safari history files
• …

Ideas about new artifacts, or even contribution to the tool are greatly appreciated. The tool can be downloaded from here and the man page is accessible here.

One example of the usage is the following scenario.  A user has opened CMD.EXE and ran the command ipconfig.  To show that the user in question was the user that actually ran the command we start by taking a traditional timeline using TSK (in this instance the machine was booted into HELIX to create the timeline):

fls -m C: -r /dev/sda1 > /tmp/bodyfile
ils -m /dev/sda1 >> /tmp/bodyfile

Then mount the drive, for instance by issuing this command:

mkdir /mnt/analyze
mount.ntfs-3g -o ro,nodev,noexec,show_sys_files /dev/sda1 /mnt/analyze

Now the suspect drive is mounted as a read-only so we can inspect some of the artifacts found on the system.

cd /mnt/analyze/WINDOWS/Prefetch
log2timeline -f prefetch . >> /tmp/bodyfile

We start by navigating to the Prefetch directory, which stores information about recently started programs (created to speed up boot time of those processes) and run the tool against the Prefetch directory.  The output is then stored in the same bodyfile as the traditional file system timeline.  Then we navigate to the user that we are taking a closer look at to examine the UserAssist (stores information about recently run processes by that user) part of the user’s registry.

cd /mnt/analyze/Documents\ and\ Settings/USER
log2timeline -f userassist NTUSER.DAT >> /tmp/bodyfile

Now we have incorporated information found inside a particular user in the bodyfile.  Let’s examine the timeline a little bit closer, use the tool mactime from TSK to create a timeline

mactime -b /tmp/bodyfile > /tmp/timeline

We can the see part of the output below:

User running CMD and ipconfig

If we examine the timeline we can now see that on Sunday July 19th at 14:25:46 the user USER ran the command CMD.EXE as displayed in the UserAssist part of that particular user’s registry file.  Then few seconds later, or at 14:25:50 the command IPCONFIG.EXE was accessed according to the traditional timeline. And then we see that a .pf file (inside Prefetch directory) is created at 14:25:53, we also see that according to the Prefetch file the command has been executed six times, and the last time it was executed was at 14:25:50 (so we know that the update of the access time did not come from someone opening the file or otherwise modifying the access time, it was really executed).

Other examples of usage would include reading LNK files to include the information found there inside the timeline.  Take for instance all the documents found inside the folder “C:\Documents and Settings\USER\Recent” that stores information about recently opened documents by that particular user.  If we read the content of that directory and include that into our timeline, for instance by issuing this command:

cd /mnt/analyze/Documents\ and\ Settings\USER\Recent
ls -b *.lnk | xargs -n1 log2timeline -f win_link >> /tmp/bodyfile

We then recreate the timeline and examine the document “Not to be seen document.txt”, which is a document that this particular user should not have read.

Timeline Analysis

If we examine the timeline above we see that at 20:23:22 on Jul the 31th the Prefetch file NOTEPAD.EXE-336351A9.pf is created, suggesting that NOTEPAD.EXE has been opened.  Then at 20:23:27 we see that both the M (modified) and A (access) timestamps have been updated (these timestamps are found inside the shortcut file itself), suggesting that the file was opened at that time using most probably NOTEPAD.EXE.  The reason why we don’t see NOTEPAD.EXE in the Prefetch timeline is that it is run again later, at 20:23:49 (which is the last time it was used).  The shortcut file itself was created at 20:23:38, which is after it had been opened, according to the information found inside the LNK file itself.

This shows that it is important to also examine the artifacts found on suspect systems and include them in the timeline analysis.

I haven’t had a time to publish a blog for a while but I wanted to let people know of the new tool I just wrote (still a Beta release), log2timeline.

This is a collection of Perl scripts (with one front-end) that parse different log files and artifacts (mostly Windows artifacts for now) to create a body file that can be modified into a timeline using tools such as mactime (from TSK).

The tool’s web site is log2timeline.net and from there you can download the tool and take a look at the man page.  The web site is quite rough, not a very pretty one, but does what it is supposed to do.  I will start publishing blogs on this site showing the usability of the tool, how it can be used to assist investigators to build a more accurate timeline of events. But for now, I will just post the link to the tool and let you test it out.  The man page of the tool as well as the README file should explain the functionality as well as future plans for the tool.