NIST report clarifies evaluation of forensic tools for mobile devices
Connecting state and local government leaders
Ensuring the quality of tools used and work done in forensic analysis is critical, and NIST has issued a report that could help in he task of validating them.
Mobile devices can be important sources of information in investigations, but ensuring the accuracy of forensic tools used to extract information from them can be a challenge.
Analysts at the National Institute of Standards and Technology describe the situation as the "forensic tool spiral" in the introduction to the newly released Interagency Report IR 7617, "Mobile Forensic Reference Materials: A Methodology and Reification," a methodology for creating reference materials and populating mobile devices to validate forensics tools.
“New versions of forensic tools are issued regularly by a tool manufacturer to provide new features, broaden the range of existing functions, and correct identified problems," the report authors wrote.
Each new version should be tested to validate the accuracy of its results, by loading known reference material onto a target device for extraction. But, “populating a device is time consuming and prone to error, especially if done manually,” the authors wrote. “The situation often creates an impasse — the improvements to the tool would benefit the work being performed, but no convenient time is immediately available to validate the tool.”
This can create a temptation to use tools that have not been popularly validated, which can lead to problems in an investigation.
“The predicament could be alleviated greatly, if validation could be expedited, possibly by devising a means to populate mobile devices readily with reference test data, to create reference material for use in tool assessment,” the authors wrote.
The new report is an attempt to provide that means.
Assessments of forensic Subscriber Identity Module tools that NIST performed in 2005 and 2007 found a number of inaccuracies, including:
- Information could not be acquired from certain SIMs through the supported interface.
- The remaining number of attempts was not provided.
- Data was rendered inconsistently in displays and reports.
- Recovered data entries were truncated when displayed.
- Recovered numeric data and English characters were not always decoded or translated correctly.
- European and Asian character sets used in phone book and message entries were not properly decoded.
- Certain EMS messages were completely missed or their content unable to be recognized and rendered correctly.
- New versions of a tool occasionally failed to perform as well as a previous one.
Much of the report focuses on the SIM, used in many call phones, because virtually all forensics tools deal with the identity modules and it is a relatively mature area. But the wide range of available character sets and limited memory of the modules can complicate the creation of reference material for validating the tools. Other challenges include:
- Encoding quirks – Lapses in coverage of special cases, can occur.
- Irregular content – Common conventions may not be followed by all carriers, creating a need for special treatment to compensate.
- Unexpected content – Content can be correct, such as zero length messages or deleted concatenated messages that have some segments overwritten, but unexpected and handled incorrectly.
- Poisoned content – Since some content is under the subscriber’s control, it may be purposefully changed to create problems for a forensic tool.
- Database consistency – Any mechanism a tool uses to translate codes into meaningful labels, needs to be kept up-to-date over time (e.g., when carriers merge or are acquired).
The scheme presented by NIST uses the Extensible Markup Language to represent the data.
“XML is a well-known and well-defined standard, similar to HTML, and users are likely to have some familiarity with it,” the report says. “Schema-sensitive XML editors are widely available, and many of them are free. These editors make it fairly easy to modify existing or create new reference test data conforming to the schema definition, once some familiarity is gained with the data elements.”