IR and forensic talk

I know I haven’t been really active on the blog lately (really not written a thing) but I wanted to talk about the new release of log2timeline. So version 0.61 was released few days ago. It mostly contains some bug fixes (at least on my behalf). The only real changes that I did was to add an input module to read the log2timeline CSV format. I also added  a bash_completion script that makes it easier to use the tool in the bash shell.

However this release does hopefully mark a shift in the development of the tool. This release has seven new input modules, six of which that were donated to me. And I’ve got one more module that will be part of the next release. This is at least to me very exciting news, since that means that other people are starting to use the tool and find it to be useful enough to add modules to it, and I hope that this shift in development will continue

I would like to thank an anonymous donator that contributed five input modules and Willi Ballenthin that contributed another module to the tool, his second one. And then also to John Ritchie that contributed a module that parses the Firefox cache files, although it hasn’t yet been distributed with the tool, sent just slightly too late for me to add it in this release. And I hope that people will continue to contribute modules to the tool.

I know that documentation has been lacking for potential developers. I hope that I will find the time soon to create such documentation, making it easier for people to contribute modules. In the mean time, I suggest looking at previously developed modules and to download the source code and look inside the “dev/” directory. There you can find templates for new modules which can assist in the creation. However better documentation is hopefully on its way

Well… I was supposed to give a talk at the SANS EU forensics summit about log2timeline but due to our lovely volcano in Eyjafjallajökull (which some people might have heard mentioned lately, although few can really pronounce it correctly) there were no flights to the UK… meaning that although the airport here in Iceland was opened (ironically) I and others couldn’t get to the summit… so it was decided to postpone it until next September.

But anyway, since the EU summit was postponed I wanted to release my slides so that at least people could have seen what I was supposed to talk about… and during the presentation I was about to announce the release of the new version, or version 0.50 of log2timeline.  But since the summit was postponed I decided to withhold the new release until I’ve completed few more tweaks and features into it.  So I’m still planning to release the new version ASAP, and when I do I will publish the slides as well.

I’ve just published version 0.41 of log2timeline, for a full list of the changes read the changelog.  This upgrade is a recommended upgrade since it contains several bug fixes as well as enhancements to the tool.  I’ve added new input modules for: Google’s Chrome History, Opera History, Firefox Bookmarks, and Windows Event Logs (EVTX). I’ve also added a new output module, CEF, for the Common Event Format as designed by ArcSight as well as improving few other input modules (more on that later).

In my last post I talked about Opera history files as well as the bookmark feature of older versions of Firefox.  Since I’ve added support for the bookmarks features in older versions of Firefox (the ones that still store their bookmark information in the bookmarks.html file) I decided to include those information in the newer versions of the browser as well.  As of version 3+ of Firefox it no longer stores bookmark information inside the bookmarks.html file.  Instead it stores them in the places.sqlite database, the same one that contains the browser history.  Therefore I upgraded the firefox3 input module to include information about bookmarks, which are stored inside the moz_bookmarks table as well as in inside the moz_places table.  The SQL command used to pull out information from the bookmarks is the following:

SELECT moz_bookmarks.type,moz_bookmarks.title,moz_bookmarks.dateAdded,
moz_bookmarks.lastModified,moz_places.url,moz_places.title,
moz_places.rev_host,moz_places.visit_count
FROM moz_places, moz_bookmarks
WHERE
 moz_bookmarks.fk = moz_places.id
 AND moz_bookmarks.type <> 3

There is one field in the moz_bookmarks table that is of special interest, that is the “type” field.  There are three different bookmark types:

• 1 = A bookmark (URL)
• 2 = A bookmark folder
• 3 = Separator

The above SQL command returns all values from the moz_bookmarks table (except separators) that have any corresponding fields inside the moz_places table.  This means that the SQL command in fact only returns bookmarked URL’s, not folders.  So another query is made to get the necessary information about bookmark folders:

SELECT moz_bookmarks.title,moz_bookmarks.dateAdded,moz_bookmarks.lastModified
FROM moz_bookmarks
WHERE
 moz_bookmarks.type = 2

This SQL command extracts all the dates associated with the bookmark folders.  But there are other tables within the places.sqlite database that might contain date objects, that is the table moz_items_annos.  This table contains additional information about bookmarks, that is annotations that are made to bookmarks.  The table stores the time when an annotation was added to a bookmark as well as when it was last modified.  The SQL command used to extract this information from the places.sqlite database is:

SELECT moz_items_annos.content, moz_items_annos.dateAdded
,moz_items_annos.lastModified,moz_bookmarks.title,
moz_places.url,moz_places.rev_host
FROM moz_items_annos,moz_bookmarks,moz_places
WHERE
 moz_items_annos.item_id = moz_bookmarks.id
 AND moz_bookmarks.fk = moz_places.id

An example output of the newly upgraded firefox3 input module is the following:

log2timeline -f firefox3 -z local places.sqlite  | grep Bookmark

...
0|[Firefox3] User: smith Bookmark Annotation: [milw0rm exploits and 0day
exploits database] to bookmark [milw0rm] (http://www.milw0rm.com/)|0|0|0|0|
0|1195573631|1195573631|1195573631|1195573631
...
0|[Firefox3] User: smith Bookmark Folder [Bookmarks Menu]|0|0|0|0|0|
1218738203|1218738203|1195573631|1195573631
...
0|[Firefox3] User: smith  Bookmark URL SANS London 2008 (http://www.sans.org/london08)
[london08] count 0|0|0|0|0|0|1218784170|1218784170|1218784170|1218784170

I’ve also upgraded the flash cookie or Local Shared Object (sol) input module considerably.  The older version was not built to support many of the common flash cookies out there so the new version should implement a parser for every known type of objects there are.  Although I’ve seen some flash cookies that the input module is not capable of parsing that have considerably different binary structure. These files might be an older version of the standard and the current version of the sol input module is unable to parse them (and so are every other SOL editors/parsers that I’ve seen).  I will not include any information about the structure of flash cookies in this post, it will be reserved for a later post.

I’ve also decided not to create all my blog posts on this site and then re-post some of them on the SANS forensics blog. Instead I will post some of the blog posts solely on the SANS blog while others will only be here.  On that spirit I wrote a post about Google’s Chrome browser which can be read here. My blog post about flash cookies will also be posted on the SANS forensics blog site.

I’ve been working on a new version for log2timeline for a while now, and I finally managed to complete some testing on the new code.  There are some significant changes to the way that log2timeline works in the new version, 0.40. Some of them are:

• All timestamps are now normalized to UTC
• The GUI, glog2timeline, has been updated so that it is feature compatible with log2timeline’s CLI front-end.
• timescanner has been further developed so that it now can parse all the artifacts that log2timeline is capable of.

Full list of the changes can be seen in the changelog.

The reason why the timestamps have now been normalized is the fact that some timestamps are stored as UTC while others use the operating system’s timezone settings.  This might not be such a big problem when using the log2timeline CLI, since it only takes one file at a time and produces a body file.  However the investigator had to knew that this particular file was either stored in the local timezone or in UTC.

The real problem arises with the use of timescanner.  When timescanner is used a directory is recursively searched for all parsable artifacts.  This means that the tool parses both the artifacts that have timestamps in UTC as well as those stored in the native timezone settings, and stores them in the same body file.  This causes the timestamps to vary and causes problems during analysis.  For this reason all timestamps are now normalized so that the output is in UTC.  This means however that the investigator now needs to provide the timezone settings of the suspect machine both to the timescanner/glog2timeline/log2timeline tool as well as mactime (if that tool is used to convert the body file to a working timeline).

The new version of the tool has been tested only on a few test cases, so if you find any bugs in it or have comments, don’t hesitate to contact me, kristinn

Just a quick post. I just updated the log2timeline code and released version 0.20beta of the tool.

The new version includes separation of the output logic to a output plugin. Only two plugins are now supported, mactime and mactime legacy (older versions). I also fixed some problems with the old code that I found and added support for reading Firefox 3 places.sqlite database file.

The code can be downloaded from the tool’s web site: log2timeline.net