Wednesday, November 20, 2013

Windows Forensic Live CD

Previously, making Windows based Forensic Live CD was not for everyone, mostly due to the amount of tinkering involved. WinXP and Win7 based Live CD's also have problems with writing a Windows drive signature to write-protected drives.

Mini-WinFE project has changed this.  Creating a Forensic Live CD with Mini-WinFE is done in a few mouse-clicks. Windows 8 and 8.1 also appear not to write a drive signature to the wire-protected disk.
From my experience Windows 8.1 Enterprise based Live CD has some issues when adding custom programs to it. The Win 8.1 Pro version works perfectly well.

The boot time is about a minute longer compared to Linux based Live CD's but you get driver and app flexibility with Windows.

TrueCrypt is missing in the default app selection. I had to spend a half an hour to fix it.
Below are scripts to add TrueCrypt 7.1a to the Live CD.

TrueCrypt must be downloaded first and extracted, not installed on the machine (though it may work also, but I haven't tested it)

TrueCrypt.script must be placed to \Mini-WinFE\Projects\WinFE\Programs folder

MD5 (TrueCrypt.script) = 383c5a68888e258f0954c009f813b3ed

To add TrueCrypt to the program menu, download and replace bblean1.17.1.script located at \Mini-WinFE\Projects\WinFE\Shell.Then.End

MD5 (bblean1.17.1.script) = 75115b21edf70501fe329cb911c80e66

Then just follow the instructions to create your Forensic Live CD and you are done. 

Monday, June 24, 2013

Advanced Cyber Threat Environment

APT or Advanced persistent threat is usually associated with governments or organised groups of hacktivists.

It is no secret however, that in the Digital Age Organised Crime has established its own presence in cyberspace. Conventional Street gang’s strength depends on their access to disposable foot soldiers willing to take the greatest risks.

Organised crime specialising in cybercrimes recruit hackers in place of foot solders. Countries with high unemployment, low wages, immature legislation and incompetent law-enforcement are breeding grounds for them. While these groups can become a gun-for-hire and be involved in APT, they normally operate on “the cash must flow” principle. This means, they must move on to another target when unsuccessful at hacking for a ‘reasonable’ period of time. The effort and persistence usually depends on the complexity of hacking or potential reward.

Having access to the skilful hackers makes these groups capable of launching sophisticated attacks, but these attacks are less persistent compared to hacktivism motivated or .gov sponsored attacks. The problem is that organised cybercrime groups are attracted to the potentially high reward targets. As a result, the different and uncoordinated groups of hackers constantly attack these targets, creating an Advanced Cyber Threat Environment (ACTE) for successful businesses and financial institutions.

Smaller and less successful businesses are also operating in ACT Environment in these countries *. In the Wild West era, it used to be a Colt, but these days DDoS attacks are often invoked by the competition as its most convincing argument. The effective DDoS attack protection relies on expert knowledge and good understanding of the company’s core business, not software or hardware. Small businesses don’t normally have the capacity.

I haven't specifically named the countries with Advanced Cyber Threat Environment and left this for you to decide.

Wednesday, March 27, 2013

Fruity Shingles

Identifying similar documents

MD 5 hashing technique has been used in Computer Forensics for years now. The primary uses are to insure integrity of evidence and also to identify identical files. Around 2006 Jesse Kornblum has published a paper on "fuzzy hashing" (official name is "context triggered piecewise hashes") algorithm capable of identifying near identical files on a binary level. I remember my excitement at the time and today I still often use this technique sifting through the clutter etc.

Thanks to the advancements in Data Mining and Information Retrieval fields, computer forensics professionals now have another tools at their disposal. It is capable of identifying near duplicates and chained near duplicates based on the context of a document, not just a binary level. The algorithm is called w-shingling or just shingling. Shingling does not attempt to understand the meaning of the text like some Natural Language Processing algorithms do, so its is language agnostic. It also works with the context of a document regardless of formatting (.pdf, .dot, .eml etc). 

The algorithm may have a significant impact on the investigator's workflow because: 
a) it is very effective at reducing the amount of irrelevant files, and
b) great at finding documents that are similar to the files, already identified as being relevant.

Some times we hear about admissibility problems caused by the use of black box algorithms in eDiscovery, most often associated with another cool technology "predictive coding". Well, shingling is not part of this legal battle and the algorithm is not a "black box". In fact, most of us already using it on a daily basis when searching the Internet. The variations of the algorithm are used by search engines to ensure that the original content gets returned in response to our keyword query and the duplicates are omitted. 


Shingling is an effective and efficient method for comparing the sets of shingles in files containing text.  As I previously mentioned, shingling doesn't rely on linguistics.

As once said by Albert Einstein “If you can't explain it to a six year old, you don't understand it yourself.” Taking this literally, the algorithm extracts the plain text from a document and performs the following:

a) removes all characters except letters and digits and puts everything into lower case*
b) splits the text into tokens (overlapping groups of words, hence its name “shingling”)
c) compares the sets of shingles generated from the documents to check how similar two documents are

Here is an example of w-shingling where w = 2 words:

Text: "Apples Are Not Oranges."


Shingle size can vary, however w-shingle size 1 would produce a bag of unordered words, which can be found in many dissimilar documents. So, essentially by shingling we capture the relationship between the words. Once we created shingles, we need a method to compare them.

Jaccard similarity

Paul Jaccard (1868 - 1944) was a professor of botany and plant physiology. He developed and published the Jaccard index of similarity in 1901. The method used to calculate the distribution of various plants in the alpine zone. It measures similarity between sample sets. Today the method is used in a wide range of disciplines including biology, genealogy, mathematics, computer science and we can now add computer forensics to this list.

Lets take two baskets of fruits, basket A and basket B

Both baskets A and B have a set of common fruit items, called the Intersection (denoted as ).

Click to enlarge

A set of unique items in either baskets A and B is called the Union (denoted as )
When two sets have one or more elements in common, these elements counted only once in their union set.

Click to enlarge

Jaccard similarity index (lets denote it as Sim) is calculated based on the number of items in the intersection of A and B divided by the number of items in the union of A and B. The result ranges from 0 (with no elements in common) to 1 (identical items).

\[Sim(A,B) = \frac{|A ∩ B|}{|A ∪ B|}\]

In the above example we have:

$Sim(A,B) = \frac{|A ∩  B|}{|A ∪  B|} = \frac{3}{10} = 0.3$

Notes and notations: Objects separated by commas, inside parenthesis (A, B, C, D, E) denote an ordered set, or (A, B) ≠ (B, A). If the objects are in inside curly brackets {A, B, C, D, E}, they are unordered, or {A, B} = {B, A}
In mathmathics, a value between |  | denotes the absolute value of a real number, which is always the non-negative value. Example | -3 | = 3.  The cardinality uses the same notation. We mean the later in our example,  where a number of elements in  |A ∩  B| is divided by a number of elements in |A ∪  B|.

Here is another example:

Lets consider two sets A and B, where:

A = {0, 1, 2, 3}   and    B = {1, 2, 3, 4, 4, 5, 6, 7}

$Sim(A,B) = \frac{\{1, 2, 3\}}{\{0, 1, 2, 3, 4, 5, 6, 7\}} = \frac{3}{8} = 0.375$

Python has a built-in class to calculate the union and intersection. Calculation of Jaccard similarity index can be done in just a few lines:

IntersectionAB = len(A.intersection(B))
UnionAB = len(A.union(B))
SimAB= IntersectionAB / float(UnionAB)

Jaccard Distance or how dissimilar two sets are, can be calculated as:

DistanceAB = (len(A.union(B)) - len(A.intersection(B)))/ float(len(A.union(B)))

An implementation of Jaccard distance is also available in Python Natural Language Toolkit and could be implemented as follows:

from nltk.metrics.distance import Jaccard_distance

A = {0, 1, 2, 3}

B = {1, 2, 3, 4, 4, 5, 6, 7}
DistanceAB = jaccard_distance(A,B)

Jaccard similarity is simple yet effective method but not without limitations:
  • Doesn't account for the frequency of the query term occurrence
  • Unique words or combinations of words are more informative compared to commonly used one
  • Longer documents produce more hits compared to short documents. I have seen a modified formula $Sim(A,B) = \frac{|A ∩ B|} {\sqrt{ | A ∪ B |}}$ been used for the length normalisation
Shingle size should be more than 1 word; take into account the length of the document and ideally the amount of commonly used words in the document. Its size should be selected to be sufficiently large to make sure the low probability of an accidental collision in the selected document (s) [collision means similarity].

* Other considerations when implementing shingling are capitalisation, white spaces, punctuation, stop words, using natural language processing to identify nouns, verbs and synonyms etc.

When dealing with large sets of data, storing generated shingles becomes a challenge. To reduce it's size, a hash function can be used instead of the actual strings. Obviously, we would need something more suitable than MD5 hash. Actually, we want the algorithm that functionally is opposite to MD5 and closer in the outcome to "fuzzy hashing".  "The min-wise independent permutations algorithm" or in short MinHash is used for those purposes. I am going to leave MinHash for some other time, as this isn't necessary for understanding the algorithm of finding near-duplicate documents.

Currently, most commercial computer forensic tools are missing this future, though I can see some snippets of the functionality being implemented in the latest FTK Lab edition.  By expanding its functionality, NUIX 4.2 has become a new player in Computer Forensics field and brought from eDiscovery a new set of useful tools. My favourites so far are:

  • solid support for various email formats (NSF, PST, EDB. etc.); 
  • fast and reliable indexing; 
  • and of course shingling, the ability to export shingle lists for use across the cases similarly to NSRL or MD5 hash sets, being able to identify near-duplicates and also find chained near-duplicates. 

Chained near-duplicates

Sunday, July 15, 2012

A quick note on Fraud <-- from the trenches

Just finished an interesting investigation, where millions of dollars have been stolen by a sales person. It turned out that the company has KPI (Key Performance Indicators) based on volume of sale, not how much profit sales team makes for the company. This approach breeds all kinds of corruption.

In this particular case CCleaner and Eraser have been used 4 times before I got the computer. The guy simply didn't think of automatic Apple backups, that were made every time he connected his precious iPad to his work computer.

Lately, I have noticed that it has become more frustrating to navigate the web. Adds have been pushed to my screen from every imaginable place. What's more annoying is that many are showing up before the content of a page that your were urgently looking for, with a little button in some obscured place allowing you to skip or fast forward the add. I wander how many annoyed or naive customers actuall click on this kind of adds and if these adds are doing more damage than good for the advertiser.

To me, this particular advertising model is not dissimilar to the above-mentioned case with all the consequences arising therefrom.

Wednesday, July 4, 2012

Miscellaneous things


Windows 7 is finally replacing Windows XP in both, private and corporate areas. According to StatCounter Windows 7 passed 50% threshold in June this year. I have been using Windows 7 almost from day one and started using this OS as a main forensic platform since release of SP1. I found that Windows 7 is more sensitive to hardware changes compared to Windows XP and occasionally would simply refuse to boot after changing settings in motherboard or adding new hardware.

I still use Dell Optiplex 755 for research and development. 8GB of Ram and Quad Core CPU handling most tasks at acceptable speeds. Last week I reinstalled Win 7 OS and this week decided to add two 2TB drives configured in RAID-0. I went to BIOS and changed Drive Operation mode from default AHCI to RAID and configured these two HHD's in Intel Storage Raid controller as RAID-0.

The OS refused to boot. I remembered how sometimes Windows XP would go into 'BSOD' and Advanced Host Controller Interface(AHCI) mode had to be switched off in BIOS. Obviously the issue was related to AHCI/RAID. Win 7 automatic repair option didn't help and I went online looking for a solution. It only took me 2 two minutes to find the fix. I disconnected two RAID-0 drives, changed back to AHCI mode and booted Windows 7. I them edited two registries and changed their VALUE date to 0, changed back to RAID mode and Voila, everything works again.



Don't drop your Thunderbolt cable

Untitled 2

I have been holding back on Thunderbolt technology due to its price and lack of available storage
devices. My focus this year was on USB3. Adding USB3 drivers to WinPE Forensic Live CD for
example is easy to do and Express cards are cheap and extremely useful when imaging laptops that
have no USB3 interface.

Thunderbolt is still expensive technology, even cables are $50 plus. The technology is very promising
though and gaining popularity. Thunderbolt cables are expensive for a good reason.

The aren't just a bunch of interconnected copper conductors anymore. To be able to sustain 10Gbps
bidirectional data transfer rate these 'wires' currently have four integrated circuits at both ends.
Transivers, microcontrollers, 3V power management and voltage regulation chips and 15V power
supply are built into the wire making it a very sensitive and advanced piece of hardware.

Monday, May 28, 2012

A short X-War story.

Around two years ago I got a new job. Part of my responsibility was to build the firm's Computer Forensic practise almost from scratch. On my first day at work I was out imaging a dozen of computers, and then brought the acquired images for processing into a room which later became our computer forensics lab. Dell Optiplex 755 and EnCase were the only tools available at the time and the current investigation urgently demanded computer forensic reports/results. Just as a side note: I have been in the industry for many years, have MCSE certifications and knew well how to install and properly configure a forensic workstation and tools.

Twenty-four hours later I was nowhere with these images, still dealing with constant EnCase crashes. Downgrading EnCase version or calling tech support was of no help.  Using open source tools was not an option due to the time constrains and to make things worse I had access to only a very slow Internet connection.

A gave a call to X-Ways and arranged an urgent delivery of X-Ways Forensics dongle. It has arrived in a couple of days. In just a few hours I had the information start flowing to the investigators. About six days (and nights) later I had all the reports done. X-Ways Forensics was rock solid with not a single crash.

Currently X-Ways Forensics has been in a very active development stage, with new features being added almost on a weekly basis. Mind you, X-Ways is already a very advanced tool with many unique features that not yet available in EnCase or FTK. Volume Shadow Copy is a good example. The tool is also often more accurately interpreting the data compare to other mainstream forensic tools. I just read Mike’s post regarding disks using 4k sectors.  Mike in his recent post mentioned X-Ways as the only forensic tool able to correctly interpret info from such disks (EnCase 7 might work as well, so may be some folk will actually start using it :-).  This is consistent with my experience and very illustrative.

Looking at the latest features of X-Ways, I wonder if the team of developers at X-Ways ever sleep. Just added support for VMDK snapshot images, support for NK2 Outlook auto-complete, IE travellog files, metadata extraction from manifest.mbdx and manifest.mbdb iPhone backup files. The most significant addition for me personally is a plug-in to run Python scripts as X-Tensions for X-Ways forensics.  Did I mention that you actually hear back from their technical support?

Monday, April 23, 2012

USB Flash drive Serial Numbers - "UNIQUE"?

Formatted USB flash drives (a.k.a. thumb drives etc) have Volume Serial numbers generated when the new filesystem gets created. The algorithm depends on a file system and OS. Volume Serial number can easily be changed via hex editor at locations:

FAT 12/16 - 4 bytes at offset 0x027
FAT 32      - 4 bytes at offset 0x043
NTFS         - 8 bytes at offset 0x48

or by using a myriad of free tools that can be found on the Internet. Volume Serial numbers are important from the forensic investigations stand point and there have been plenty of good material written on this topic. The most prominent in my view are written by Craig Wilson, Rob Lee and Harlan Carvey.

Unlike Device Serial Numbers, Volume ID's get captured by all forensic imaging tools. Device Serial Numbers however have been considered by computer forensic practitioners as more reliable and "Unique" artefacts. In Windows there are several places when Device Serial Numbers get recorded/logged. USBStor  registry key and Windows log files: Setupapi.log on Windows XP or on Vista and above are the most obvious one. It is also a well known fact that when a USB flash drive has no serial number, the system assigns to the device its own number with an ampersand symbol as the second character of this serial number.

The question is, how "UNIQUE" these Device Serial numbers are?
Well, as it turns out, these numbers are not necessary unique. There could be several reasons for this.

1. There is a tool that gamers are using to spoof device serial numbers called PB DownForce. It is capable of temporary changing the device serial number. The serial number can be changed to a random or predefine serial number.

This wont fool (see picture below) tools like USBDeview, but the software that rely on Operating System to obtain the serial number will fall for it.

2. USB drives serial numbers are meant to be at least 12 valid characters, represented as a UNICODE string.  "The last 12 digits of the serial number shall be unique to each USB idVendor and idProduct pair" according Universal Serial Bus Mass Storage Class paper.

   Valid Serial Number Characters

        Numeric                        ASCII
0030h  through 0039h      "0" through "9"
0041h through 0046h       "A" through "F

These requirements have not been adopted as the mandatory standard and a lot of manufacturers use shorter and in many cases identical numbers on their cheaper drives.

3. Big labels do use "unique" serial numbers, especially on their upper class, higher capacity USB devices. Still some reuse serial numbers every 6th million times as in case with one of the popular USB storage manufacturer I had to deal with.

4. Devices can be FAKE. On eBay there are plenty of fake 'false capacity usb flash drives', including brand name counterfeits such as 16GB Kingston, 32GB Sandisk etc. Serial numbers on these devices can be ether, all identical or generated at random.

5. User can change the device serial number accidentally or on purpose.  There are many tools, mostly used to fix faulty USB flash drives, capable of changing the device serial number. FixFakeFlash Inspectortech website is a good place to learn more about fake USB devices and tools capable of changing many parameters on the USB device including the serial number, ability to create, encrypt, hide or write protect certain area on the device.

The above-mentioned tools are designed to work with different USB flash drive controllers and you of course must have the right one to be able to reprogram the device.

Names of memory controllers can be coded in the original (Factory set) Serial Number. For example some Kingston's devices in13th position of the serial number have a letter A, B, E, C or F:

Kingston DataTraveler 200 USB Device SN: 001A92053B6ABB4131340023

A        - SkyMedi
B or E - Phison
C or F - SSS

To my knowledge similar tools are available for the memory controllers listed below:
  • Alcor
  • Ameco (MXTronics)
  • Chipsbank
  • iCreate
  • ITE tech
  • Netac
  • OTI
  • Phison
  • Prolific
  • Skymedi
  • SMI (Silicon Motion)
  • SSS (Solid State System)
In addition to USBDeview there is another excellent tool called ChipGenius (by Chinese Developers at that provides a lot of useful information about a USB Device. The tool can be used to check pretty much all types of USB devices including external hard drives and MP3 players, detect fakes and view the device controller vendor.

It displays chip model, manufacturer, revision number, VID/PID, interface speed, protocol, serial number and media type information.

Finally, unlike Volume Serial Numbers most forensic imaging tools don't capture a Device Serial number. The only exception to this rule I know are Tableau imagers. Both hardware (TD1 & TD2 duplicators) and Software (TIM a.k.a. Tableau's High Performance Software Imager) include the Device Serial Number in the acquisition log automatically (but not in the image itself).

Speaking about Tableau devices, a new generation of TD duplicator TD2 is looking really sexy. TD1 has been used by my team quite extensively. The new version "can optionally include USB, SCSI and SAS suspect drive" and what is even more exciting is the ability to image 1:2 or how Tableau called it "Twinning" support. According to the specifications it also supports EnCase v7 .ex01 (AES encrypted) format. I definitely going to order one of these very shortly.

Wednesday, April 11, 2012

HELLO - Almost missed it.

Computer Forensic tools are rapidly improving and make forensic examinations easier for the masses. Only a qualified forensic practitioner however can reliably produce consistently good results.
For example at present no computer forensic tool can properly detect, search and index text in the Unicode escape sequence. I have recently been working with the image containing some iPad sqlite3 backup files and found an extremely important piece of evidence almost by accident. Well, not exactly by accident, just have been thorough really.
\u0048 \u0045 \u004c \u004c \u004f means HELLO when you convert it from the Unicode-escape, which Apple tends to use quite extensively for recording non Latin characters. Python comes to rescue (once again) with its built-in sqlite3 library to pull the data and .decode('unicode_escape').

A quick script solved the problem, so I get some free time to finally watch "George Harrison: Living in the Material World" this weekend which has been on my to-do list for a couple of months now.

And to make it clear, the important piece of evidence I found wasn't "HELLO" word   

Monday, February 27, 2012


Sharing information on the net has some risks associated with it. "..if you rear yourself against it, you shall fall, you shall be bruised, you shall be battered, you shall be flawed, you shall be smashed." Dickens, Bleak House (1853) Yet still, I would rather see more information and a healthy discussion or argument about the issue, than seeing nothing. I am glad to see more computer forensic blogs popping out, some of the are really great and some are just excellent. Periodically I get a chance to speak to a very knowledgeable people. These people have a lot to learn from, but they become algophobic of a very thought of putting snippets of their knowledge or ideas online.

Yes, there are risks if you haven't verified your information or your assumptions were wrong. You very well may end up in a situation like this snowman.

There might be some people out there showing off their "knowledge" without doing a thing themselves to contribute to Computer Forensic community. These people usually look and behave like this snowman :-)

Remember 'Star Thrower story' by Loren C. Eiseley where a young girl was at a beach full of washed after storm starfish. She was picking them up and throwing them back into the ocean. When she was told that she can't possibly make any difference bacuase there are thousands of them around, she picked up another one and said "Well, I made a difference to that one!".

Unfortunately I don't post often, simply because I am currently working in a country where computer forensics discipline is in its infancy and only one university recently launched a computer forensic course. There is a lot of work  in educating, training and explaining besides working the cases, which leaves me with a very little time for any research or blogging.

You cant say I am not trying though :-)

.. and yes, lots of snow around.

Friday, February 17, 2012

PFX – Personal inFormation eXchange

A password and PFX file are needed to open encrypted e-mail messages, whose content is enveloped and attached as smime.p7m. PRTK does a good job at cracking passwords, but some PFX files have different headers which PRTK would not recognise. Chilkat Python Modules come pretty handy in this situation. Modules come with a fully-functional 30-day trial and need to be purchased for use beyond this period or for commercial purposes. I wrote a script, which is based on one of the Chilkat module examples to allow a dictionary attack on PFX and p7m encrypted message. The code is quick and dirty, but gets the job done.
You will need your.p7m encrypted message, your.pfx file and a good ASCII formatted wordlist with .txt; .dic or .lst file extension.

A sample code is provided for illustrative purposes only and  "AS IS" without any warranties of any kind. :-) The code has not been thoroughly tested under all conditions, but should work fine if you know what 're you doing. Here is the LINK to it. It should work fine on Windows and maybe on Lin/Mac machines as well (some modifications may be needed). The script relies on Chilkat modules, which must be installed prior to running the script. Instructions are on pyPFX project home.

Thursday, November 24, 2011

a couple of newly discovered tools

It's been an extremely busy autumn for me. Whilst running around, I came across a couple of useful tools.

SAFE (System Acquisition Forensic Environment) is Windows PE boot disk with built in software write blocking. I use Enterprise version, which requires a dongle only to start up the environment. The dongle then can be removed to start up the next machine. A bootable USB can also be created with SAFE USB Creator. There are several tolls listed as officially SUPPORTED by ForensicSoft, but plenty of other tools can also run just fine in this environment. To get the ability to image over the network I put F-Response on the Live CD as well and found it to be working rather well.  SAFE has some problems with recognising Unicode file names when opening with OpenOffice for example and some other minor bugs. Win PE is based on Windows 7 32-bit and works well with most hardware.

Another Windows based GUI Forensic Imager has been released in beta. This time from GetData.  It has a very simple interface, works in a portable mode and supports  DD, AFF and E01 image formats. It also converts from one format to another. I wonder if it remains free after it is out of beta. 

Wednesday, August 3, 2011

SSD - TRIM, Encryption, Formating and Fragmentation

Operating System identify Solid State Drives by querying the hard drive for its rotational speed. To be precise it is done by identification of nominal rotation rate as described in AT Attachment – 8 ATA/ATAPI Command Set (ATA8-ACS).
Word 217
0000h -
rate not reported
0001h -
Non-rotating media (SSD)
0002h-0400h -
0401h-FFFEh -
Nominal media rotation rate in rotations per min (rpm)
7200rpm = 1c20h 5000rpm = 1388h 10 000rpm 2710h

If 0001h value is returned, Windows 7 for example turns on TRIM support and disables defragmentation. Furthermore, to reduce the frequency of writes and flushes, Windows 7 in addition to boot and application launch prefetching also disables services such as ReadyBoost and Superfetch. As far as I am aware Windows XP or Windows Vista cannot differentiate SSDs from hard drives. The following file systems are known to be TRIM supported by its respective Operating Systems: NTFS, HFS+, EXT4, Btrfs. Here I should mention that modern Linux and Apple OSX support TRIM commands as well. TRIM functionality can also be implemented independently of the operating system. The O&O Defrag for example enables TRIM operations for FAT32 and exFAT formatted SSD’s.

I know that many forensic folks are still wondering how OS’s, file systems and SSD controllers talk to each other to make TRIM work. Louis Gerbarg did an excellent job of explaining and demystifying the process.

It should be noted that Windows 7 sends the TRIM command to the SSD not only when file gets deleted or partition gets formatted, but in several other instances as described in Support and Q&A for Solid-State Drives blog post.

"The Trim operation is fully integrated with partition- and volume-level commands like Format and Delete, with file system commands relating to truncate and compression, and with the System Restore (aka Volume Snapshot) feature."

A quick format is all that is required to trigger the TRIM command on SSD and all data will be erased (zeroed out). Speaking about formatting, there has been not much difference between the Quick and Full format options in pre-Vista Windows machines. The only difference between the two was that full format also scanned for bad sectors.  The data could still be recovered from formatted drives. Since Windows Vista a full format erases all data and writes zeros and completely destroying the old data. The same applies to Windows 7 and my tests confirmed this. 

TRIM can be enabled and disabled manually. In Windows 7 to check TRIM status, as Administrator in the command prompt window, enter the following:

fsutil behavior query disabledeletenotify

DisableDeleteNotify = 1 Windows TRIM commands are disabled
DisableDeleteNotify = 0 Windows TRIM commands are enabled

The following command enables TRIM fsutil behavior set disabledeletenotify 0 and fsutil behavior set disabledeletenotify 1 disables it.

To my knowledge TRIM is not yet supported in RAID volumes. Recently there has been some confusing on this topic in relation to Intel Rapid Storage Technology supporting TRIM for RAID volumes. Intel had to publish a correction that TRIM is only supported in AHCI and RAID modes for drives that are not part of a RAID volume.

Not all SSD’s support the TRIM command; some manufacturers do not even recommend enabling TRIM. Sandforce and OCZ recommend against enabling TRIM in the Mac OS (due to Apple's implementation of TRIM) and discourage using TRIM on controllers with internal low-level compression (due to the way they operate/built).

TRIM + Encryption, a topic worth its own cookbook, so I am going to only lightly touch on it. In my previous post I have mentioned that Apple OS X Lion “FileVault 2” enables whole-disk encryption. It is certainly a big step forward compared to “FileVault 1”; however this needs to be clarified a bit. “FileVault 2” is VOLUME based encryption.  For example NTFS, FAT/FAT32 or exFAT partitions located on the same drive will not be encrypted. A recovery partitions also cannot be encrypted by “FileVault 2”. TRIM is believed to be supported on “FileVault 2” encrypted drive. The TRIM command also works on NTFS file system encrypted with Bitlocker and TrueCrypt . TrueCrypt has issued several security warnings in relation to Wear-levelling security issues and the TRIM command revealing information about which blocks are in use and which are not. (Trim Operation Link & Wear-Leveling Link) PGP WDE doesn’t support TRIM, but I remember someone has mentioned that with CLI is possible to encrypt only used sectors. It is likely that the same security issue would arise as in case of TrueCrypt.

Sunday, July 31, 2011

The Mighty Lion

Snow Leopard 10.6 wasn't much of a problem from the forensics perspective and left paws imprints all over the snow. It had no TRIM enabled by default and FileVault was not particularly difficult to deal with. Advanced users could install TRIM for their SSD drives by using TRIM Enabler 1.1 but this wasn't wide spread. Apple OS X Lion 10.7 came and the game has changed.

The new OS adds support for the TRIM command and it is turned ON by default. TRIM allows OS-level garbage collection and also assists with wear-levelling and fragmentation, as well as reducing write amplifications and improves random writes speed. Basically if an operating system supports TRIM, delete really does mean delete, not just flagging space as available.

OS X Lion also introduces "FileVault 2", which instead of merely encrypting user home folders, now offering "Full Disk Encryption". Upon upgrading existing users are offered to upgrade to "FileVault 2". Old FileVault, lets call it "FileVault 1" is also supported but only for existing users of "FileVault 1". The new encryption method uses XTS-AES 128-bit encryption. When "FileVault 2" is enabled, a user is presented with the option to create a recovery key.

WARNING: You will need your login password or a recovery key to access your data. A recovery key is automatically generated as part of this setup. If you forget both your password and recovery key, the data will be lost.

Recovery key: CCQP-DDA3-XDSF-5656-UHGX-MTN8

Additionally, Apple now provides with an option to store the recovery key with them, which I am sure will be useful for both, forgetful users and law-enforcement.

Monday, July 18, 2011

Safeboot with EnCase or FTK

Both (current versions) of EnCase and FTK work with Safeboot Full Disk Encryption 4.x.
EnCase has to be 32 bit version (not 64 bit). According to Guidance Software support people Safeboot 4.1 or higher versions are not supported by EnCase. In reality Safeboot 4.1 decryption works just fine with EnCase 6.18 as long as one follows the detailed instructions.

FTK 3 officially supports SafeBoot Version 4.x and Version 5.x as well as McAfee Endpoint Encryption Version 6.x. There is no '32 bit only' limitations because there is no need to install SafeBoot Tool or anything extra.

Access to the SafeBoot server is requred when working with both EnCase and FTK.There is no need to export/copy out any files for decrypting with FTK. For Safeboot versions 4.x and 5.x the decryption key can be obtained by runing SbAdmCl.exe command line tool. It's location can vary from version to version on the Safeboot server.

SbAdmCl.exe -AdminUser:admin -AdminPwd:password -command:GetMachineKey -Machine:Machinename

To extract decryption keys for a group of computers the same command can be issued with  -Group:* instead of -Machine:Machinename

The command should return 32 bit Encryption Key(s) that can be entered in FTK when the encrypted evidence files are added to the case.

In McAfee Endpoint Encryption Version 6.x the key is exported from the server by using ePO (ePolicy Orchestrator). Check "Exporting the recovery information file from ePO" section of McAfee EETech User Guide for details. Once the .xml file is exported, a base64 key located between < key > and < / key >  needs to be copied, decoded and converted to hex. The easiest way to accomplish the task is to utilise this online "Base64 -> hexadecimal string decoder", which should produce the decryption key required by FTK.

UPDATE: 16 August 2011
 EnCase Version 6.19 just has been released. The new version now provides support for McAfee Endpoint Encryption 6.0.

Monday, June 13, 2011

No trust in a single tool.

"If the only tool you have is a hammer, you tend to see every problem as a nail."
Abraham Maslow

More and more often I find myself working on a case with at least two forensic tools simultaneously. Depending on a task I select EnCase and X-Ways or FTK and X-Ways in pairs.

All three are great and one is better than another at certain tasks.I like working with EnCase to analyse registries, automate things with enscipts or searching and bookmarking hits in unallocated space. FTK is best with emails and has excellent ‘indexed’ searching capability. X-Ways Forensics is simply fast and reliable.

There is no point in doing ALL operations with a pair of these tools. There are always several the most important pieces of evidence supporting the hypothesis that need extra attention. This is especially true when confirming the absence of certain evidence.

I don’t just use two tools in parallel, in addition I attempt to utilise different methods to confirm the facts. This becomes some sort of Devil's Advocate Peer Review Activity.

Lately, forensic tools became more complex and attempting to provide more interpretation for the sake of convenience. Not surprisingly, I frequently observe different interpretations by different tools and have to dig dipper to find the true.

Although I often use a bunch of open source or free tools like Harlan’s RegRipper or Mandiant’s Highlighter etc., having another full featured forensic tool provides an additional layer of protection. Several times I had a situation when the main tool would start constantly crashing, or be unable to process certain types of evidence in the middle of examination. Sounds famialiar? When time is limited and vendor’s technical support is slow or sometime useless, having a back up tool ready to go is as good as gold.

Selecting the right tools for different investigations requires a good knowledge of forensic tools in your arsenal. For example, Lotus Notes is very popular in the corporate environment, with over 140 million corporate licensees sold worldwide. EnCase would normally work with NSF files and handle emails quite well. You will need FTK, or some other solution, to handle Lotus Notes databases, because EnCase …. well, may be EnCase 7 will do a better job. X-Ways Forensics can’t handle NSF at all. For the sake of completeness I should mention here that since Lotus Notes version 8.5 Databases are now called Applications.

Obviously one needs to be trained on using all of these tools and this might not be economically possible for small organisations or Rookie examiners. In this case there are Open Source Resources/Tools that each examiner must become proficient with and have them ready to go. The new book by Cory Altheide and Harlan Carvey called Digital Forensics with Open Source Tools should provide you with the necessary knowledge and insight.

Sunday, June 5, 2011

Most computer forensic examiners Need Shrinks

Many computer forensic specialists sooner or later get exposed to potentially psychologically harmful material. Images or (worse) videos of people being tortured and killed; children being exploited and raped are often encountered by forensic examiners. Some have only occasional exposure, and some have to constantly work with such material due to the nature of their work. The exposure causes all sorts of problems from stress and loss of productivity to more serious psychological traumas.

The above also applies to private and corporate forensic examiners who often accidentally locate offensive images or videos. What are the ways to minimise negative impacts of exposure to such material?

Prevention is better than cure.
It is technically difficult to completely insulate all personnel from the exposure. The only logical choice is to adequately prepare specialist for such situations by introducing mandatory introductory programs. These programs need to be specifically designed to deal with exposures to potentially harmful material and possible reactions to such exposures. Most importantly new computer forensic specialists must be put through the program before they walk in to the lab.

As part of occupational health and safety, career longevity and work performance initiative we are currently working with professional psychologists to develop such program for our organisation. The program is going to be integrated in the Standard Operating Procedures (SOP), and will also include mandatory reporting, debriefing and follow up. To minimise harmful effects, the arrangements are being made with psychologists to conduct debriefing within the first 24 to 72 hours after the initial exposure.

These procedures are designed to equip computer forensic personnel with knowledge, skills and professional assistance to enable them to cope with exposures to offensive graphics. As an additional benefit, the program  may also assit staff in dealing with other stressful situations. These steps are also designed to insure productivity and retention of the highly trained forensic specialists.

Thursday, May 26, 2011

Oh mama - my iPhone is no longer secure!

ElcomSoft guys are offering " near-instant forensic access to encrypted information stored in iPhone devices" ...even if its hardware encrypted.  Here is a LINK to the the press release. Good job.

I hope it won't repeat destiny of COFEE. 
Relevant read from ElcomSoft's blog link1 & link2

Saturday, April 9, 2011

DDos on LiveJournal - turning crisis into opportunity.

Developing an effective incident response procedure is crucial to minimizing the impact of a security breach or DDoS attack. A good incident response plan not only helps secure the impacted infrastructure, but can also increase consumer loyalty. The recent DDoS attack on LiveJournal clearly required the use of public relations techniques, which did not appear to happen in time.

In the absence of information, the rumour mill will take over. Instead, an immediate and honest statement should clarify known details, and the information be frequently updated. The organisation must demonstrate commitment and this will be appreciated by its customers.  In case certain information cannot be released it is important to offer an explanation. By doing this the organisation appear responsive and cooperative even if not a great deal of information has been released.

The organisation also must educate all employees on use of social media during the crisis and monitor Twitter, MySpace, Facebook and other social sites. Tracking and quickly responding to the relevant conversations should help uncovering and defusing any potential crises-in-the-making.

While no organisation is immune to similar  incidents, this does not necessarily have to turn into a disaster.

Saturday, April 2, 2011

Accessing VMFS partitions

VMware VMFS is VMware Virtual Machine File System with is used by VMware ESX and ESXI servers to store virtual machine disk images (.VMDK) and snapshots. The VMDK (Virtual Machine Disk) files are equivalent to the real hard drives, except they are virtual. Many forensic tools, including EnCase can analyse VMware (.vmdk) data files or mount them (FTK Imager, Mount Imager Pro etc.). The problem is getting VMDK files out of VMFS without ESX or ESXI infrastructure. There are several solutions to this problem.

Open Source VMFS Driver was written by fluidOps in Java; it's free and allows read-only access to files located on VMFS partitions by utilising many operating systems including Windows. Java version 6 is required to run it. All you needed is to mount E01 image containing VMFS partition with your favourite tool. I used to love Mount Image Pro and Smart Mount, but people change. I am using FTK Imager v3 now for obvious reasons; it doesn't cost me anything and no pain with dongles or registrations.


Running the following command should get you into the partition via webdav interface C:\vmfs_r95>java -jar fvmfs.jar \\.\PhysicalDrive4 webdav

Next navigate to http://localhost:50080/vmf and you should see VMDK files you were after.
Correction: I forgot to put an "s" at the end of the above address. The correct address would be http://localhost:50080/vmfs Thanks Tim for pointing this out.

The world isn't perfect though and you may run into a couple of problems:

Problem 1:
You may get an error similar to this:
Exception in thread "main" VMFS FDC base not found

Problem 2:
There are several partitions inside your E01 image; some of them could be FAT12 "Hypervisor" partitions, which is enough for fluidOps driver to give up on you.

There are several ways of getting inside however. In my case I happened to have VMware Workstation installed on my machine and one of the guest OS was Ubuntu 10.10. I have added Hard Disk (PhysicalDrive4) to my Linux guest OS and started it.

vmfs-tools is yet another tool, which is "originally loosely based on the vmfs code from fluidOps" and allows read only access to VMFS file systems from non ESX/ESXi hosts.

In Linux I installed vmfs-tools by running: sudo apt-get install vmfs-tools and typed the following command: sudo fdisk –l

The above shows that the vmfs file system is located on /dev/sdb3

The next command is to mount VMware VMFS partition:
mkdir /home/a/Desktop/system and vmfs-fuse /dev/sdb3 /home/a/Desktop/system
and see what's inside.... ls -alh

I then connected (1TB USB Seagate Freeagent GO) to the virtual machine and copied the files for further analysis. DONE.

P.S. Paul Henry did a good write-up on a similar subject  here.

Saturday, January 22, 2011

GPU password cracking.

GPU acceleration has been used to crack passwords for some time now. This is due to GPU's parallel layout, which is a hip better at large-scale mathematical operations compared to ordinary CPU’s. Before, there was only nVidia with its CUDA SDK. I must admit that while I was building the lab and doing lots of administrative work, I totally missed the arrival of AMD’s Stream SDK. It appears that ATI Radeon cards are much faster at crunching the numbers, in some cases x 10 times and software developers are quickly adding support for ATI cards. I just discovered a nice blog on password cracking by Vladimir Katalov from ElcomSoft. The blog is very informative and a good read. The author  mentioned that a new version of Elcomsoft Phone Password Breaker for example already supports both nVidia and ATI cards achieving speeds  around "7,000 passwords per second on NVIDIA GeForce GTX 580, and about 20,000 passwords per second on ATI Radeon HD 5970".

Wednesday, December 15, 2010

Sleuthkit 3.2.0 on Ubuntu 10.10

Some time ago I have written a short "how-to" in relation to installing the Sleuthkit on Ubuntu. Recently I have tried to install the latest Sleuthkit 3.2.0 on Ubuntu 10.10 (32-bit) and ran into a problem when compiling it. It took me some time to figure out how to get it working.

Step 1:

sudo apt-get install libewf1 libewf-dev zlib1g-dev build-essential libexpat1-dev libfuse2 libfuse-dev fuse-utils gvfs-fuse libncurses5-dev libreadline-dev uuid-dev libssl-dev

Step 2:

Download and extract afflib 3.6.4
In terminal go to the extracted directory and run the usual
sudo make install

Step 3:

Download Sleuthkit 3.2.0 and extract it. Next I had to apply a quick fix by adding LDFLAGS link option to file located inside the extracted sleuthkit-3.2.0 directory. Adding the following line LDFLAGS="$LDFLAGS -lsqlite3 -lpthread -ldl" seems to fix the problem.

I then navigated to sleuthkit-3.2.0 directory in terminal and run

sudo make install


Tuesday, December 7, 2010

iSCSI initiator on Win 7

F-Responce (and  Helix3 Pro) both can be handy for imaging over iSCSI.  Win 7 iSCSI initiator looks slightly different to Win XP.

Typing iscsicpl and hitting enter brings the initiator.

In Discovery tab press Discover Portal. This should open another window Discover Targt Portal. Enter IP address and port (if not default) and click Advanced button.

In Advanced Settings window mark Enable CHAP log on and enter username and password as per F-Responce target configuration.

The target(s) should appear in Discovery tab.

In Targets tab there should be the drive with status indicated as Inactive.
Click connect button.

Another window will open and there will be an option to add this disk to favorite targets. It is up to you if you 'd like to do that or not. Click Advanced button.

The same proceedure here, Enabling CHAP log on and entering username and password.

The drive should be connected now.

Tuesday, November 30, 2010

Imaging SAS drives the easy way.

Every time a came to image machines with Serial Attached SCSI (SAS) hard drives, I thought about SAS writeblocker. The problem was that there were no such things available. Live CD's, F-Responce, Live Imaging, SAS to SATA Adapters (I haven't tried this one) or SAS cards were the only options. I am glad that recently Tableau came up with one such device. It is called Tableau T6es SAS and I am just about getting one. 


Many nice things have already been said about FTK Imager 3, which is certainly my tool of the year. It even works from USB Flash drive with all these nice new features for mounting image files. Just copy the folder from "C:\Program Files (x86)\AccessData\FTK Imager" onto your portable drive and you are pretty much set.

Tuesday, November 23, 2010

iOS 4.2 has arrived!

Apple iOS 4.2 Software has finally arrived making my beloved iPhone and iPad even more functional and probably introducing new bugs/vulnerabilities.  I must admit that I have lately jumped on the Apple wagon, even right now I am typing this blog on MacBook Pro :-) .   

I still do most of the forensic work on Windows machines and only occasionally utilising Linux. 
Having a busy life lately, I have Mac(s) mostly for personal use, and the main reason for choosing Apple devices for me was it's functionality, relative security and low maintenance.

I recently attended a presentation, where several current Windows vulnerabilities/hacks have been demonstrated. These little beasts were able to disable all major antivirus solutions, even when executed with 'guest' privileges. Another logical attack vector on commercial antivirus software would be an attack on it's license, for example by corrupting the license or changing the clock to the future, making AV's license expired. Several commercial products dropped it's defences in my tests straight away. 

The funniest thing was that the above mentioned presentation was given right after a computer forensic presentation by a young and very enthusiastic  person, who was questioning the need to have a forensic machine disconnected from the Internet, while performing the examination. I simply have no time or energy to deal with possible security compromises and other issues that may arise from having my forensic machine connected to the Internet. At the end of the day I have bought these Apple gadgets to safe my time for something better than constantly fixing my home Windows computer or checking firewall and security logs on my forensic machine :-) after each forensic examination.

Monday, November 15, 2010

BranchCache - Distributed Cache Mode

BranchCache is designed to solve problems with the availability of information in remote offices with slow WAN connections.

According to Microsoft BranchCache is only supported on Windows Server 2008 R2 and Windows 7 Enterprise and Windows 7 Ultimate. The technology supports two modes: Hosted Cache and Distribute Cache.  It allows data to be cached on computers in the remote branch office and is made available to other computers in the branch.

In Hosted Cache mode, the content is cached on a Windows Server 2008 R2 content server on the remote branch network. In Distributed Cache mode the content is distributed between Windows 7 client computers on the remote branch network and no additional server infrastructure is required. When distributed mode is enabled, a client computer first receives information from the BranchCache content server at the head office. The next client computer that requests the same information from the head office only receives the (small in size) content information and actual content is obtained from another client computer in the remote branch.

Files changes are monitored by using hashes. If the client is unable to locate the necessary file in his own cache, it sends requests to the local subnet via UDP protocol and then fetches it from one of  the local client computer via HTTP/HTTPS.

Not only the actual content, but the requests and 'content information' might potentially be a good source of valuable evidence.

Monday, September 27, 2010

Evidence movers

Using an evidence mover helps to transfer files around and preserve its integrity. It is also savesa lot of time on image verification after the evidence have been transferred. I have been using MicroForensics Evidence Mover (the latest version is 1.1.17) for quite some time now. It is a nice free tool. There is one little problem with this tool. When the destination drive becomes unavailable, MicroForensics Evidence Mover happily reports that all files have been successfully transferred. Unless you check for the logs and make sure that every (source) file has been listed in the log, there is a good chance that the transfer is incomplete.

Nuix Evidence Mover 2.0.21 is also free and looks and feels like the one from MicroForensics, except one little detail. The tool from Nuix actually reports that all files have been transferred OK. If the destination drive becomes unavailable during the transfer, you will not see the line similar to this one:

09/27/10 12:09:58 - All files were moved successfully

Friday, September 17, 2010

DRM protection

This pastebin
page probably has been one of the most visited place lately. Hardware Blu-Ray rippers HDfury2 and DVIMagic may soon have software competition due to the HDCP master key getting out in the wild.

Saturday, September 11, 2010


FTK 3.x "PATTERN" is using Boost C++ RegEx libraries, which is a new name for Regex++.

There are three main syntax options available for Boost: Perl, POSIX extended and POSIX Basic with Perl being default. It is good to know that FTK is definitely using Perl implementation. The exact RegEx syntax is available here.

... and yes, I am back. .. well kind of... I'm just not sure how often I 'd be able to post here.

Thursday, February 25, 2010

This blog will be updated soon!

This blog has not been updated for some time. I am planning to update it soon.

Saturday, January 30, 2010

Acronis Try&Decide

Acronis True Image Home 2010 is a backup utility that offers ability to perform full, differential and incremental backups. Be able to mount Acronis back-up image as a logical drive in read or read/write mode is also handy. Acronis True Image is more then just a backup software however. It includes Disk Cleanser, File Shredder, and System Clean-up, which wipes data stored on a hard disk, individual partitions or individual files.

The software also has a nifty feature called "Try and Decide". As the name might suggest, it is designed to give users a second life whilst they make potentially dangerous changes to the system. It is easily activated by pressing "Try&Decide" button.

When Try and Decide is activated, all the changes made be the user are recorded in an automatically created folder named "Acronis Try&Decide" on external hard drive instead of drive C. Virtualisation technology is used to "isolate your "real" operating system from changes" and there is no need to install VMware or other virtualisation software.

Try&Decide continue working after the system reboots. Upon completion, the user is presented with  options to accept or discard the changes.

After changes have been discarded and Try&Decide was stopped, the folder "Acronis Try&Decide" gets automatically deleted.

Inside "Acronis Try&Decide" folder the program creates a sub-folder that looks similar to C59FD9A9-D675-48B8-80E2-38662B09C411.  This sub-folder contains a single file where all temporary data is being stored by Acronis. Searching for hex value 4163726f746e6430 should locate this file unless it has been overwritten.

Wednesday, January 13, 2010

Knowledge - Management and Retention

Along digital forensics and information security I have always been interested in knowledge management and knowledge retention subjects. These areas are especially relevant to Information Security/Digital Forensics because these disciplines heavily rely on highly knowledgeable professionals. When such professionals leave the organisation, they create a giant gap that has to be filled.

There are several publications on this topic, many of them packed with unnecessary statistical data, useless formulas and usually boring as dry toast.

I just finished reading a book by Jay Liebowitz "Knowledge Retention Strategies and Solutions" and I was pleasantly surprised by the quality of material. This book is written to be concise and full of insights and knowledge of topic.

It is hard to disagree with the author who suggests that "younger workers are less likely to stay with one employer for more than a few years" and that a "learning organization" must develop "knowledge retention strategies so that critical knowledge does not walk out the door".

Unfortunately, I haven't seen many such organisations around, at least not in this industry. Instead, I came across many good professionals who would keep their expertise to themselves and only share the knowledge when it suits they own interests. In his book Liebowitz identifies major challenges to knowledge sharing and states that 'about 80% of knowledge management is people, culture, and process, and only 20% is technology' such as document management systems, wiki's etc. He suggests that the experts should be motivated to share their knowledge "through being recognized and rewarded". Of course this would require a competent management capable of creating the right atmosphere and build a high level of trust throughout an organisation.

The author also mentioned the knowledge-engineering paradox, which I found to be quite amusing but dead right. The knowledge-engineering paradox 'means that the more expert an individual, the more compiled his/her knowledge and the harder it is to extract that knowledge'. Recently, I was surprised when someone told me, that occasionally it is hard to get a quick technical explanation from me. I thought about it for a moment and then realised that I have to decompile this information first and only after that, translate it to a language understandable by a non technical person.

This book is a good read and should be a valuable addition to every computer forensics manager's library.