IR and forensic talk

Just recently saw a post at Slashdot about Adobe implementing private browsing in their Flash Player.  That means that when the user starts private browsing mode in their web browsers LSO files will not be stored on disk.  This is implemented in the way that during the private browser session all Flash cookies are stored only in memory, and as soon as the browser is closed they are cleared.

Why do we care about this? Well with this change we will start to see that private browsing is becoming more private (or actually private), and it will make our lives as forensic investigators more difficult, since we cannot simply examine Flash cookies to determine the users browsing history (at least partially).

I just posted a blog post in the SANS forensics blog about the structure of LSO files and a quick view of how log2timeline parses it to extract timestamps. I’m not going to repeat that post on this site, so if you would like to know more about the binary format of LSO files, please read the blog post.

Recently there have been a lot of discussions about creating a standard for timeline analysis. Currently log2timeline relies upon the good old mactime format for it’s output (although it is possible to use several different output mechanism), a standard that was created for filesystem timelines.  Although it works great for its original purpose It might not be the most optimal output mechanism when incorporating timestamp information from other sources which is one of the reasons why this push for a new standard has been discussed. The structure of log2timeline will be changed soon to separate the internal structure away from the emphasis on mactime and move to a more neutral approach, and perhaps change the default output mechanism to something like TLN.

With the move to a more neutral approach more logic will be moved into the output modules, meaning that it will be easier to make the description text (which every output module includes) can be more descriptive and does not need to repeat information that might be contained within the output itself (such as TLN which includes a source field, why repeat the source in the description field?)

I just wanted to write a short post about the upcoming SANS European Digital Forensics and and Incident Response Summit that will take place in London on the 19th and 20th of April.  I encourage everyone that has the chance to attend since there are some very interesting talks, such as; Jesse Kornblum’s talk about fuzzy hashing, Keith Foggon’s discussions about trends and techniques and Lee Whitfield’s Windows Shadow Volumes presentation.

I will also be there, talking about log2timeline.  The title of my talk is Mastering the Super Timeline – log2timeline style.  After the talk I will participate in a tool talk panel, so there is your chance to pound me with some difficult questions…

The abstract of my talk is:

Traditional timeline analysis can be extremely useful yet it sometimes misses important events that are stored inside files on the suspect system (log files, OS artifacts).  By solely depending on traditional filesystem timeline the investigator misses context that is necessary to get a complete and accurate description of the events that took place.  To achieve this goal of enlightenment we need to dig deeper and incorporate information found inside artifacts or log files into our timeline analysis and create some sort of super timeline. These artifacts or log files could reside on the suspect system itself or in another device, such as a firewall or a proxy.  This talk will focus on the tool log2timeline, which is a framework built to parse different log files and artifacts to produce a super timeline in an easy automatic fashion, designed to assist investigators in their timeline analysis.

So the talk will contain some of the work in my upcoming Gold paper, titled “Mastering the Super Timeline With log2timeline” (did someone notice the similarities between the titles).  The paper should be published soon, at least before the summit.

I’ve just published version 0.41 of log2timeline, for a full list of the changes read the changelog.  This upgrade is a recommended upgrade since it contains several bug fixes as well as enhancements to the tool.  I’ve added new input modules for: Google’s Chrome History, Opera History, Firefox Bookmarks, and Windows Event Logs (EVTX). I’ve also added a new output module, CEF, for the Common Event Format as designed by ArcSight as well as improving few other input modules (more on that later).

In my last post I talked about Opera history files as well as the bookmark feature of older versions of Firefox.  Since I’ve added support for the bookmarks features in older versions of Firefox (the ones that still store their bookmark information in the bookmarks.html file) I decided to include those information in the newer versions of the browser as well.  As of version 3+ of Firefox it no longer stores bookmark information inside the bookmarks.html file.  Instead it stores them in the places.sqlite database, the same one that contains the browser history.  Therefore I upgraded the firefox3 input module to include information about bookmarks, which are stored inside the moz_bookmarks table as well as in inside the moz_places table.  The SQL command used to pull out information from the bookmarks is the following:

SELECT moz_bookmarks.type,moz_bookmarks.title,moz_bookmarks.dateAdded,
moz_bookmarks.lastModified,moz_places.url,moz_places.title,
moz_places.rev_host,moz_places.visit_count
FROM moz_places, moz_bookmarks
WHERE
 moz_bookmarks.fk = moz_places.id
 AND moz_bookmarks.type <> 3

There is one field in the moz_bookmarks table that is of special interest, that is the “type” field.  There are three different bookmark types:

• 1 = A bookmark (URL)
• 2 = A bookmark folder
• 3 = Separator

The above SQL command returns all values from the moz_bookmarks table (except separators) that have any corresponding fields inside the moz_places table.  This means that the SQL command in fact only returns bookmarked URL’s, not folders.  So another query is made to get the necessary information about bookmark folders:

SELECT moz_bookmarks.title,moz_bookmarks.dateAdded,moz_bookmarks.lastModified
FROM moz_bookmarks
WHERE
 moz_bookmarks.type = 2

This SQL command extracts all the dates associated with the bookmark folders.  But there are other tables within the places.sqlite database that might contain date objects, that is the table moz_items_annos.  This table contains additional information about bookmarks, that is annotations that are made to bookmarks.  The table stores the time when an annotation was added to a bookmark as well as when it was last modified.  The SQL command used to extract this information from the places.sqlite database is:

SELECT moz_items_annos.content, moz_items_annos.dateAdded
,moz_items_annos.lastModified,moz_bookmarks.title,
moz_places.url,moz_places.rev_host
FROM moz_items_annos,moz_bookmarks,moz_places
WHERE
 moz_items_annos.item_id = moz_bookmarks.id
 AND moz_bookmarks.fk = moz_places.id

An example output of the newly upgraded firefox3 input module is the following:

log2timeline -f firefox3 -z local places.sqlite  | grep Bookmark

...
0|[Firefox3] User: smith Bookmark Annotation: [milw0rm exploits and 0day
exploits database] to bookmark [milw0rm] (http://www.milw0rm.com/)|0|0|0|0|
0|1195573631|1195573631|1195573631|1195573631
...
0|[Firefox3] User: smith Bookmark Folder [Bookmarks Menu]|0|0|0|0|0|
1218738203|1218738203|1195573631|1195573631
...
0|[Firefox3] User: smith  Bookmark URL SANS London 2008 (http://www.sans.org/london08)
[london08] count 0|0|0|0|0|0|1218784170|1218784170|1218784170|1218784170

I’ve also upgraded the flash cookie or Local Shared Object (sol) input module considerably.  The older version was not built to support many of the common flash cookies out there so the new version should implement a parser for every known type of objects there are.  Although I’ve seen some flash cookies that the input module is not capable of parsing that have considerably different binary structure. These files might be an older version of the standard and the current version of the sol input module is unable to parse them (and so are every other SOL editors/parsers that I’ve seen).  I will not include any information about the structure of flash cookies in this post, it will be reserved for a later post.

I’ve also decided not to create all my blog posts on this site and then re-post some of them on the SANS forensics blog. Instead I will post some of the blog posts solely on the SANS blog while others will only be here.  On that spirit I wrote a post about Google’s Chrome browser which can be read here. My blog post about flash cookies will also be posted on the SANS forensics blog site.

I’ve been working on a new version for log2timeline for a while now, and I finally managed to complete some testing on the new code.  There are some significant changes to the way that log2timeline works in the new version, 0.40. Some of them are:

• All timestamps are now normalized to UTC
• The GUI, glog2timeline, has been updated so that it is feature compatible with log2timeline’s CLI front-end.
• timescanner has been further developed so that it now can parse all the artifacts that log2timeline is capable of.

Full list of the changes can be seen in the changelog.

The reason why the timestamps have now been normalized is the fact that some timestamps are stored as UTC while others use the operating system’s timezone settings.  This might not be such a big problem when using the log2timeline CLI, since it only takes one file at a time and produces a body file.  However the investigator had to knew that this particular file was either stored in the local timezone or in UTC.

The real problem arises with the use of timescanner.  When timescanner is used a directory is recursively searched for all parsable artifacts.  This means that the tool parses both the artifacts that have timestamps in UTC as well as those stored in the native timezone settings, and stores them in the same body file.  This causes the timestamps to vary and causes problems during analysis.  For this reason all timestamps are now normalized so that the output is in UTC.  This means however that the investigator now needs to provide the timezone settings of the suspect machine both to the timescanner/glog2timeline/log2timeline tool as well as mactime (if that tool is used to convert the body file to a working timeline).

The new version of the tool has been tested only on a few test cases, so if you find any bugs in it or have comments, don’t hesitate to contact me, kristinn

Just a quick post. I just updated the log2timeline code and released version 0.20beta of the tool.

The new version includes separation of the output logic to a output plugin. Only two plugins are now supported, mactime and mactime legacy (older versions). I also fixed some problems with the old code that I found and added support for reading Firefox 3 places.sqlite database file.

The code can be downloaded from the tool’s web site: log2timeline.net

Update 1

Timeline analysis can be extremely useful during any investigation.  Although traditional file system timeline can be very helpful it sometimes misses important events that are stored inside files.  These events might be crucial to the investigation or at least provide a better view of the events that really occurred on the suspect system. So to get the big picture, or a complete and accurate description we need to dig deeper and incorporate information found inside artifacts or log files into our timeline analysis. These artifacts or log files could reside on the suspect system itself or in another device, such as a firewall or a proxy (or any other device that logs down information that might be relevant to the investigation).

Unfortunately there are few tools out there that can parse and produce body files from the various artifacts found on different operating systems to include with the traditional filesystem analysis. Harlan Carvey has been working on some scripts for the Windows platform to accomplish this, such as regtime.pl to create a body file from the registry. Usually these tools are build specifically to parse a file/artifact that is of a particular format (such as a tool just to produce a body file from restore points). I’ve released some tools like that, as well as H. Carvey and others. I know of one attempt to create a framework to correlate different artifacts into a timeline, a project called Ex-Tip, by Mike Cloppert. There is a GCFA gold paper describing the framework. This project was started in May 2008, but hasn’t been maintained since then. Instead of extending that project I decided to start my own, that is to add a tool that can correlate information found inside different log files and artifacts into the traditional timeline analysis. I wanted to be able to easily integrate this tool into already existing tools that deal with timeline analysis, so I chose to output all timelines in a mactime body format, to be used with the tool mactime from TSK (The SleuthKit). This tool is called log2timeline and already supports incorporating seven different artifacts into the timeline.

In other words, this tool has been created to use artifacts and log files found on suspect systems (and others) in a timeline analysis to assist the investigator so that he can more easily see the “big picture”. That is to be able to build a more accurate timeline of the events that have occurred and when (and in which order).  Such a tool has to have a wide range of support for different log files and artifacts to be useful for investigators, yet despite only being capable of parsing six artifacts today I would like to publish my first beta version of the tool for people to download and try out.  Current version of the tool parses the following artifacts:

• Prefetch directory (reads the content of the directory and parses files found inside)
• UserAssist key info (reads the NTUSER.DAT user registry file to parse the content of UserAssist keys)
• Squid access logs (with emulate_httpd_log off)
• Restore points (reads the content of the directory and parses rp.log file inside each restore point)
• Windows shortcut files (LNK)
• Firefox history (for version 3.+)
• Windows Recycle Bin (INFO2)

Although not nearly enough support for different artifacts, at least it is a start.  Future versions will support at least:

• Event Logs
• Index.dat files (IE History)
• FF files (FF History older version)
• ISA text export
• Squid access log with httpd_emulate equal to on
• Cisco ACL entries
• Linux syslog
• pcap dump files
• Mac OS X artifacts
• Other Linux artifacts
• Opera and Safari history files
• …

Ideas about new artifacts, or even contribution to the tool are greatly appreciated. The tool can be downloaded from here and the man page is accessible here.

One example of the usage is the following scenario.  A user has opened CMD.EXE and ran the command ipconfig.  To show that the user in question was the user that actually ran the command we start by taking a traditional timeline using TSK (in this instance the machine was booted into HELIX to create the timeline):

fls -m C: -r /dev/sda1 > /tmp/bodyfile
ils -m /dev/sda1 >> /tmp/bodyfile

Then mount the drive, for instance by issuing this command:

mkdir /mnt/analyze
mount.ntfs-3g -o ro,nodev,noexec,show_sys_files /dev/sda1 /mnt/analyze

Now the suspect drive is mounted as a read-only so we can inspect some of the artifacts found on the system.

cd /mnt/analyze/WINDOWS/Prefetch
log2timeline -f prefetch . >> /tmp/bodyfile

We start by navigating to the Prefetch directory, which stores information about recently started programs (created to speed up boot time of those processes) and run the tool against the Prefetch directory.  The output is then stored in the same bodyfile as the traditional file system timeline.  Then we navigate to the user that we are taking a closer look at to examine the UserAssist (stores information about recently run processes by that user) part of the user’s registry.

cd /mnt/analyze/Documents\ and\ Settings/USER
log2timeline -f userassist NTUSER.DAT >> /tmp/bodyfile

Now we have incorporated information found inside a particular user in the bodyfile.  Let’s examine the timeline a little bit closer, use the tool mactime from TSK to create a timeline

mactime -b /tmp/bodyfile > /tmp/timeline

We can the see part of the output below:

User running CMD and ipconfig

If we examine the timeline we can now see that on Sunday July 19th at 14:25:46 the user USER ran the command CMD.EXE as displayed in the UserAssist part of that particular user’s registry file.  Then few seconds later, or at 14:25:50 the command IPCONFIG.EXE was accessed according to the traditional timeline. And then we see that a .pf file (inside Prefetch directory) is created at 14:25:53, we also see that according to the Prefetch file the command has been executed six times, and the last time it was executed was at 14:25:50 (so we know that the update of the access time did not come from someone opening the file or otherwise modifying the access time, it was really executed).

Other examples of usage would include reading LNK files to include the information found there inside the timeline.  Take for instance all the documents found inside the folder “C:\Documents and Settings\USER\Recent” that stores information about recently opened documents by that particular user.  If we read the content of that directory and include that into our timeline, for instance by issuing this command:

cd /mnt/analyze/Documents\ and\ Settings\USER\Recent
ls -b *.lnk | xargs -n1 log2timeline -f win_link >> /tmp/bodyfile

We then recreate the timeline and examine the document “Not to be seen document.txt”, which is a document that this particular user should not have read.

Timeline Analysis

If we examine the timeline above we see that at 20:23:22 on Jul the 31th the Prefetch file NOTEPAD.EXE-336351A9.pf is created, suggesting that NOTEPAD.EXE has been opened.  Then at 20:23:27 we see that both the M (modified) and A (access) timestamps have been updated (these timestamps are found inside the shortcut file itself), suggesting that the file was opened at that time using most probably NOTEPAD.EXE.  The reason why we don’t see NOTEPAD.EXE in the Prefetch timeline is that it is run again later, at 20:23:49 (which is the last time it was used).  The shortcut file itself was created at 20:23:38, which is after it had been opened, according to the information found inside the LNK file itself.

This shows that it is important to also examine the artifacts found on suspect systems and include them in the timeline analysis.