IR and forensic talk

Well, I’ve finally decided to release version 0.50 of log2timeline.  Lot of things have changed since version 0.43, although there is only one new input module introduced to the tool, we will get to that later.  I just wanted to go over some of the changes made to the tool.

First of all the verification phase has been slightly changed in all input modules to make it slightly more optimized, thus making timescanner run considerably faster on a mounted image.  Secondly, which is perhaps the biggest structural change, is the modification of the timestamp object as it is called within the framework.  The timestamp object is basically a Perl hash that contains information about the parsed line or timestamp within a file being parsed by the tool.  The timestamp object is created by the input module, and then used by the output module to create the appropriate output.  The old structure of the timestamp object was too dependent on the mactime output, making it difficult to create new output modules, without repeating some information.  So the timestamp object was completely changed, making it independent of any particular output method, making the output modules need to process the data a bit more, but instead make the output a lot more intuitive.

Other very important changes is the addition of code contribution in the tool, for the very first time someone actually contributed code to the tool (something I’m hoping is a new trend, not just an once in the lifetime of the tool).  One new input module for parsing the output of both psscan and psscan2 modules in volatility was added by Julien Touche as well as the new timescanner_threaded application that was contributed by Ben Schmitt.  One of the new things about version 0.50 was supposed to modify timescanner so it would be a threaded application, making it considerably quicker than the older single-threaded one.  With version 0.50 the very first threaded version is released as a proof of concept, and it is not recommended for real use, since it really isn’t any faster than the single-threaded one and sometimes it skips printing some of the timestamps.  So do not use it for anything else than to test it, and report bugs.  I’m hoping that in the next version we will have this fixed, perhaps it needs to be completely rewritten to work properly, perhaps there are just few bugs that we need to sort out.

Another change that has to be mentioned is the ability of timescanner to choose which modules to use.  The older versions had the mentality of either use every input module (timescanner) or just a single one (log2timeline).  Version 0.50 introduces the possibility to manually select which modules are loaded up by timescanner and used for the recursive scan.  The option -f of timescanner is the key here in choosing the appropriate modules.  The option can be used in the following way:

• -f module1,module2,module3
  The first option is just to list all the modules that you want to use, comma separated.
• -f=-module1,module2,module3
  The second method is to use the minus (-) signal to indicate which modules you want to skip.  However it should be noted that if you use the – signal you have to prepend it with an equal sign ( -f=-module1).  This tells timescanner to load up all available input modules EXCEPT the ones that are listed up.
• -f list
  The third option is to use a pre-defined lists of modules to use.  These pre-defined lists are simply text files that contain the names of the modules to use.  The current lists that are included with the tool are:

web
 chrome, firefox3, firefox2, ff_bookmark, opera, iehistory, iis,

winvista
 chrome, evt, exif, ff_bookmark, firefox3, iehistory, iis, mcafee, opera, oxml,
pdf, prefetch, recycler, restore, sol, userassist, win_link, xpfirewall,

winxp
 chrome, evt, exif, ff_bookmark, firefox3, iehistory, iis, mcafee, opera, oxml,
pdf, prefetch, recycler, restore, setupapi, sol, userassist, win_link,
xpfirewall,

There are several other changes that have been made, so read the full changelog to see a list of all changes.  Two new output modules have been added to the tool, TLNX which is simply a TLN output in a simple XML format.  The other new module is the BeeDocs, which is a timeline visualization tool that runs on Mac OS X.  The output module saves a tab delimited text file that can be imported directly into the tool.

I was also supposed to give a talk about log2timeline at the SANS EU forensic summit, that got canceled because of our lovely volcano here in Iceland.  I promised to release my presentation as soon as I would release the new version, so here it is. The presentation contains among other screen shots of the BeeDocs output module as well as some better description of the timestamp object and the inner structure of log2timeline.

I recently got the question of how to start with your timeline analysis.  And usually when someone finally asks you the question, you know that there are quite a lot of others that have absolutely no idea how to go about such analysis yet somehow don’t have the guts to ask.  Therefore for those that have never done any timeline analysis before or just want to get a better clarification on the meaning of the fields provided in the timeline, etc, here is my mini guide to get you started in your quest of timeline analysis, and who knows, this might be the first post in a series of similar ones.

First of all you need to create the super timeline. That should be the first step, without it there isn’t much to analyze.

We don’t want a simple filesystem timeline, which although can be revealing just doesn’t tell us give us enough overview of what happened on the system.  So we would like to start by extending it into a super timeline, using few tools to extract as much information as we possibly can.  And since there is still not a single tool to do all that for us, at least until I’ve added the functionality of these tools into log2timeline, we will have to make due with these great instructions on how to create a super timeline.

Since we are adding information using few tools we need to use a common output method, and in this case we used a mactime output to create the bodyfile and then change it into a CSV file using the mactime tool from the Sleuthkit (as the instructions go over, step-by-step).  Now we are ready to import this into a spreadsheet application of our choice to start our analysis.  But first thing first… what do all of these fields mean and especially in the context of a super timeline?

The format of the mactime body file starting from version 3.x is an ASCII file, pipe delimited, which is structured in the following way:

MD5|name|inode|mode_as_string|UID|GID|size|atime|mtime|ctime|crtime

The mactime body file was created to properly represent timestamps from filesystems (since that is what TSK does) so you can see there are four timestamps in each line, even though there aren’t always four recorded timestamps in every filesystem. We still use this format to describe entries that have originated from other sources than files, that is information extracted from within log files, even though some of the fields have no real meaning and quite possibly have the potential to make the analysis more difficult to understand.

A quick explanation of all the available fields in the mactime body format is perhaps needed.  Some of the information found here is taken from TSK wiki.

• MD5
  This is the MD5 sum of the file, something that isn’t really used at all, but kept there just in case someone would want it (very time consuming to calculate the md5 sum for each file) – although it is possible to populate this field using log2timeline.
• name
  This is the name and path of the file
• inode
  Although there is no notion of inodes in most filesystems this field’s name is still inode.  The value of this field refers to the metadata address, which differs depending on the filesystem in question.  In the FAT context this refers to the FAT number and in NTFS this is the MFT number, etc…
• mode_as_string
  Again, here is something that refers to the *NIX way of representing file access settings, dwrxrwxrwx is the default standard for representing file access rights in *NIX.  If you see the letter, then that access is defined, otherwise it is filled with -.  The first letter represents the file type, if we take a closer look at the Wiki for the TSK we see the following definition of the file type field:

  -: Unknown type

  r: Regular file

  d: Directory

  c: Character device

  b: Block device

  l: Symbolic link

  p: Named FIFO

  s: Shadow

  h: Socket

  w: Whiteout

  v: TSK Virtual file / directory (not a real directory, created by TSK for convenience).

  The majority of the entries will be either ‘-’, ‘r’ or ‘d’, others are mostly *NIX focused. The next three letters represent write/read/execute, which are the three access rights you can set on a file in a *NIX system.  You see that this is repeated three times, the first set is the access settings of the owner of the file (user settings), the next the group settings (each file has only one group and one user) and last you have settings for everyone else.

  In other words, the mode:
  -rwx—r–
  Means that this is a file that can be read, modified and executed by it’s owner. All other members of the group that the file belongs to cannot do anything with it, that is to say they have no access rights. And everyone else, those that do not belong to the group and are not the owner, can read the file but not execute it nor modify it.

• UID
  This is the User ID for the owner of the file.
• GID
  This is the Group ID for the group permission of the file.
• size
  The size of the file
• atime
  Mactime uses the MACB method of representing timestamps.  And the meaning of each of these timestamps differ between filesystems, so I will use a very generic description here and then show you a more detailed one later.
  This is the file’s last access time.
• mtime
  This represents the last time it was modified.
• ctime
  This represents the time when the file was changed.
• crtime
  This represents the time the file was created.

So we need to take a closer look at the MACB (modified, accessed, creation, birth) definition which is used.  Since each filesystem contains there own definition of the timestamps we really can’t generalize and say that these timestamps have the same meaning in each context.  So we need to take a look at it from a different perspective, I will just include a table from the Sleuthkit web site:

MAC Meaning by File System

If we take a closer look at this, you can start to see where this form does not properly describe timestamps extracted from other sources. Although the timeline to be analyzed in the spreadsheet application contains only one timestamp per entry, it will contain the MACB definition of the timestamp in question. Each line in the CSV file contains the following fields:

Date,Size,Type,Mode,UID,GID,Meta,File Name

In other words, you have the date, the size of the file, the mode, user ID, group ID, the META or inode number and the “File Name” field.

How does a registry entry fall into these fields? It does not necessarily have a group ID, nor a user ID, and not really a size field, inode number nor a file name, etc… here comes the a bit of artistic license into play when use these fields to properly describe such events.  And another thing, a registry entry has one timestamp, called “Last Write time”… how does that fit into the MACB definition that is more geared towards filesystems?  This makes the need of adjusting the definition of MACB.

Each file has there own settings in regards to timestamps, and in log2timeline context you will see the following input modules (assuming the latest published release, which is of this time version 0.43).  this is the meaning of the MACB fields found within the timeline:

Some of these modules will include a size field definitions, while other do not, etc.  So you will see different fields populated by different modules, depending on if that particular item is really applicable. And you will see that the “File Name” field is changed for a description of the event as extracted from the artifact in question.

This of course only applies to the mactime output, yet other output mechanism share some of these fields.  In future posts I will go into more details of both other output mechanism as well as the actual analysis part, showing examples of timelines and how to interpret them.

One of the nice things about exporting the data into the mactime body file (besides the fact that you can include information from other tools) is that you can easily transform it into a CSV file for importing into a spreadsheet application.  And in the spreadsheet application you can easily hide the fields that you are not interested in, making the analysis easier. So typically what I do is to split the date part into date and time, then I hide the mode,uid and gid fields.  After that you can turn on filtering in the spreadsheet and start analyzing (or just use grep/less/vim combination). Of course this is just my own preference while doing timeline analysis, and it depends on what I’m looking for as well.

There has been some discussion lately about some limitations to timescanner in regards to the reading of timestamps in various index.dat files.  More precisely Windows decided that it would store timestamps using different timezones depending on the location of the index.dat, instead of sticking with the good old UTC format.  So for instance the history files (index.dat that is stored in the History.IE5 folder) are stored using local timestamps, while the daily and weekly history files have timestamps that are stored using both UTC and the time zone of the machine in question (all timestamps are still stored as a Windows FILETIME format).

So as a quick fix to the current release (and nightly built) I’ve just excluded the daily and weekly files from the tool timescanner.  But in the coming 0.50 release (keep mentioning that) I’ve included a more intelligent scanner, whereas I take into consideration the location of the file in question and apply the appropriate settings to the timestamps.  So timestamps that are stored in an index.dat file that have different meaning depending on their location will have the correct meaning and description in version 0.50 as they should be.

I will provide examples and more details about this new feature when I will release the tool.

Well… I was supposed to give a talk at the SANS EU forensics summit about log2timeline but due to our lovely volcano in Eyjafjallajökull (which some people might have heard mentioned lately, although few can really pronounce it correctly) there were no flights to the UK… meaning that although the airport here in Iceland was opened (ironically) I and others couldn’t get to the summit… so it was decided to postpone it until next September.

But anyway, since the EU summit was postponed I wanted to release my slides so that at least people could have seen what I was supposed to talk about… and during the presentation I was about to announce the release of the new version, or version 0.50 of log2timeline.  But since the summit was postponed I decided to withhold the new release until I’ve completed few more tweaks and features into it.  So I’m still planning to release the new version ASAP, and when I do I will publish the slides as well.

I decided to make the installation of log2timeline a bit easier, since I know that the installation of all those Perl libraries can be a burden sometimes, especially since most packaging systems don’t have all of the libraries in their repositories.  So I started out creating an Ubuntu repository that contains not only log2timeline but also the Perl libraries that are not part of the standard distribution.  All you need to do is to add the following line to the /etc/apt/sources.list file:

deb http://log2timeline.net/pub/ karmic main

Then to get apt to accept my GPG key you need to download it from here (MD5/SHA256) and make apt use it by issuing the command:

apt-key add gpg.asc

This is it… now you should be able to use the repository and simply issue the command:

apt-get update

apt-get install log2timeline-perl

And all the dependencies should be fixed and you have a working copy of log2timeline on your Ubuntu box, as simple as that. The only problem I’ve had so far is that I’m only distributing the code in the i386 and all architectures, so for those that are using a different architecture (let’s say amd64) you need to either download the “all” package or use the good old manual installation until I’ve added your architecture to the repository.

In other news, if you are using Fedora, you should be able to use the CERT repository. They are maintaining a Fedore repository for forensics tools.  So if you are using a Fedora machine, all you need to do is to follow the instructions given here and then issue

yum install log2timeline

And all dependencies will be sorted out for you. Couldn’t be simpler to get it installed.

The next step would be to make the code OS neutral and create a working Windows executable to distribute, since there are still some parts of the code that are *NIX only, such as problems with the / vs. \ and the use of temporary directories, all minor issues but need to be solved nonetheless. And it would be nice as well to add a macports package as well, to make installation easier on the Mac OS X platform, something to work on in the near future…

And check out the new post from Chris talking about timescanner. He got timescanner to run on Windows, although I can see in the output that not all of the code seem to work, such as getting the username for some of the input modules.  This is most likely to do with the / vs. \ problem (*NIX vs. WIN), so this is one of the issues I need to fix to get the tool to properly run on the Windows platform.

I forgot to mention Aftertime in my last blog post, which is a new tool to create and analyse timelines.  Rob pointed this tool to me the other day, and I’ve done some limited testing on it.  It is very easy to create the timeline, just add the image file and let it crunch through it, all point-and-click and easy.  That is nice and I’m sure some will prefer that over the CLI method of log2timeline, where you need to use the command line and know the parameters of the tool, etc.  The tool also provides a nice GUI to display the timeline, using separate colors for each source, and to create reports.  Yet somehow I got the feeling it might be easy to overlook some of the important events, especially if they are only couple of them. This might be because I’m not used to examining timelines visually like this or because it might be hard to detect a single event that is surrounded with benign ones using a visual method like this. This is something I have to test further, since I think there are a lot of benefits of being able to visualize the timeline.

Harlan Carvey posted yesterday about some of these links that you see in this post.  One of which was the addition of regtime.pl into the timeline that includes every change made to the registry.  I haven’t added that functionality into log2timeline yet, that is to parse every single registry key into the timeline.  Today I’ve only included the UserAssist key, which adds more context to the registry entries than simply dump everything there.  In the near future you will see a lot more registry entries parsed using log2timeline, where I intend to parse only specific keys to add to the timeline, parsed and put into context.  I’ve been playing around few of these entries and I hope to add in version 0.51, at least part of my thoughts on the subject.

Although I agree with Harlan that adding every registry entry into the timeline can sometimes be an overkill and drown you with events and that in some cases it might loose some context (since you are not parsing the content of the keys).  However I have to admit that in some cases it really helps you find some registry entries that you might have otherwise missed.  I know that it has helped me greatly in at least few exams that I’ve done where I used tools like regtime.pl or reglookup-timeline to create the timeline.  In those cases I had a very specific timeframe which I was looking at, making the addition not so difficult to parse through, and found evidence or settings of software that I did not know at the time was installed (since timeline analysis is often the first step I do).  That led me quickly to what I was really looking for, thereby shortening the investigation time considerably.  I’m not saying that I wouldn’t have found what I was looking for using other methods, but adding the content of the entire registry into the timeline greatly reduced the investigation time so I think there is definitely value in it.  That being said, adding modules for log2timeline that actually parse the content of some specific keys and adding context to those last write times adds more value to the timeline than simply just the last write time and the name of the key, but it will never catch everything and every little piece of software you might have installed.  One thing that I liked about Aftertime though was that you could easily put everything into the timeline and then if you didn’t like seeing all the registry keys for example you could simply exclude them from the timeline and focus on something else, so if a particular source was somehow not useful at all, you could easily exclude it from the timeline (something that can be done using awk for instance in an ASCII file, but not something that everyone perhaps likes to do).