Digital Forensics Guide

Part 4: Wider sources and hidden dataChapter 13 of 17About 13 minutes

Chapter 13 of 17

13. Reconstruction and recovery

Metadata first, carving last. The filesystem's own records are the only source of names, paths, timestamps and cluster order, so you rebuild from what survives before you ever reach for a carver. When nothing else works, fragmented files can still be reassembled by structure, if you validate every join.

The quick version

  • Reconstruct metadata before carving. Carving throws away names, paths, timestamps and allocation order.
  • Use the backups the filesystem keeps: backup GPT, MFT mirror, backup superblocks.
  • Never write a reconstruction back into the working copy. Keep it separate, with its own hash.
  • Carvers assume contiguous files. On fragmented data their output may be truncated, corrupt or right by luck.
  • A fragmented JPEG can be rebuilt by structure: markers, tables, restart markers, decoding, thumbnails.
  • A reconstructed file is a derived artefact, never simply "recovered".

Open this when: a volume is damaged, files have been deleted, or you are about to run a carver and need to know what its output is worth.

In this chapter

  1. Filesystem reconstruction (section 18)
  2. Fragmented file recovery (section 19)

18. Analysis

Filesystem reconstruction

In short: understand the filesystem as it is, damage included, and recover as much structure as the surviving metadata allows before any carving.

With a readable (or decrypted) volume in hand, metadata reconstruction comes before carving. The filesystem's own records are the only source of file names, paths, timestamps, ownership and, above all, the allocation information that says which clusters belong to which file and in what order. Carving throws all of that away; it is the fallback, not the first move.

Work through the structures in order

  1. Partition tables. Parse the MBR or GPT and compare it with what is actually on the disk. A GPT keeps a backup header and table at the end of the disk; a damaged primary can be reconstructed from it. Where the table is missing or describes a layout the disk no longer has, search for filesystem boot sectors and superblocks at their expected alignments to find volumes the table does not mention.
  2. Filesystem metadata. The boot sector or superblock, and the master structures it points to: the master file table on NTFS, inode tables and group descriptors on ext4, the object map and container superblock on APFS. NTFS keeps a mirror of the first MFT records; ext4 keeps backup superblocks in several block groups. Damage to a primary is often recoverable from these copies; record the fact of the damage.
  3. Directory structures. Walk the tree from the root, noting where it breaks. Entries that survive without a parent (orphans) are recovered by scanning metadata records directly rather than walking the tree.
  4. Allocation information. The bitmap or allocation table says which clusters are in use. Compare it with the sum of the allocations claimed by file records. Clusters in use but claimed by no surviving file, and clusters claimed by a file but marked free, both point to damage or deletion; list both.
  5. Journals. The NTFS log file and change journal, the ext4 journal and the APFS object history record recent metadata operations, including creations, renames and deletions of files whose entries no longer exist. A journal can show that a file existed, its name, its size and when it was deleted, when nothing else does.
  6. Deleted entries. A deleted file's metadata record commonly survives, marked unused, until it is reused. It still holds the name, timestamps and the list of clusters the data occupied.
  7. Slack space. The space between the end of a file's data and the end of its last cluster holds whatever was there before. Small, but it can contain fragments of earlier files and, on some systems, the remains of memory.
  8. Unallocated space. Everything the filesystem no longer claims. This is where carving operates. Examine it last, informed by everything learned above about where deleted files used to be.

Keep facts and interpretation apart

Every volume found by searching for boot sectors is reported as an observed fact (a valid boot sector at a stated offset), with the interpretation kept separate.

Why a deleted record beats a carve

If a deleted record's clusters have not been reallocated, the file is recoverable in full and in order, with its metadata. That is a far stronger result than a carve.

Choose the tools

  • The Sleuth Kit and Autopsy (open source) parse partition tables and filesystems, list allocated, deleted and orphaned entries, extract files by metadata address, report cluster runs, and extract unallocated space for carving. Their command-line tools are well suited to producing the exact, reproducible offsets a report needs.
  • X-Ways Forensics is strong on damaged filesystems, with the ability to locate and interpret lost metadata records, and on hex-level work.
  • EnCase and FTK are established commercial platforms with validated filesystem support and case management.
  • Equivalent validated tools are acceptable.

Use more than one on damage

Parsers differ in how they handle corruption, and a structure one tool rejects another may interpret. Disagreement between tools is itself a finding to record.

Record the reconstruction

Every reconstructed structure is a derived artefact. Record:

  • The surviving copies used: their offset and content.
  • The interpreter: the tool and version.
  • Each decision you made, for instance choosing the backup GPT over a primary that was partially valid.

Stop and think

Keep the damaged originals in the image untouched. Never write your reconstruction back into the working copy; keep it as a separate description or a separate derived image with its own hash.

Example

A decrypted data volume has an NTFS boot sector whose MFT pointer refers to clusters that contain zeros. The MFT mirror is intact and locates the real MFT, which is largely present. Parsing it recovers 41,000 active records and 3,200 deleted records; the change journal shows a bulk deletion of a folder of image files on a stated date, and a wiping utility's own log on the system volume records a run at the same time. The report gives the offsets, the record numbers and the journal entries as observed facts; the wiping run as recovered evidence; and "the folder was deliberately deleted using the wiping tool at that time" as a conclusion, strongly supported, with the supporting artefacts listed.

Before you move on

  • Partition table checked against the disk, backups used where the primary is damaged.
  • Allocation compared with file records, and every mismatch listed.
  • Journals and deleted records examined before unallocated space.
  • Surviving copies, tool versions and decisions recorded; originals untouched.

19. Analysis

Fragmented file recovery

In short: follow surviving cluster runs where you can; where you cannot, rebuild from the file's own structure and validate every join.

Files are fragmented when their data is stored in non-contiguous clusters. Normal use produces some fragmentation. Deliberate fragmentation, where a file is split and its pieces scattered or interleaved with other data to defeat recovery, is an anti-forensic technique that turns up in cases involving images in particular. JPEG photographs are the running example here: they are the most common subject and have the most usable internal structure.

When allocation metadata survives

  1. Read the cluster runs from the file's metadata record, deleted or not. They are listed in order.
  2. Extract the clusters in that order. The file is reconstructed directly, with its name and timestamps, however badly fragmented it is.
  3. Check each run against the allocation bitmap and against other files' runs. If a cluster has since been reallocated to another file, that part is overwritten and the result is partial.
  4. Record which clusters were intact and which were not.
Source recordClustersResult
IntactAs listedRecovered evidence.
DeletedAll still freeRecovered evidence, noted as deleted.
DeletedSome reusedPartial. Only now does reconstruction begin.

When allocation metadata is gone

Without a metadata record, the only sources of information are the file's own content and the content around it. Conventional carvers find a file's header signature and copy a contiguous block from it to a footer signature or a maximum size:

  • PhotoRec recognises a large number of formats, is aware of some internal structure, and can be run against a raw image, a partition or an unallocated-space extract.
  • Scalpel and Foremost carve on header and footer signatures defined in a configuration file, quickly and reproducibly.
  • bulk_extractor scans an image for features (email addresses, URLs, EXIF data, embedded JPEGs including thumbnails) without regard to filesystem structure. Its carved JPEGs and EXIF records are a useful independent source.
  • X-Ways and the commercial suites carve by signature and, for some formats, by structure.

All of these assume, by default, that a file's data is contiguous from header to footer. Against a deliberately fragmented image they produce one of three things:

  • Truncated: the file ends at the first fragment boundary.
  • Corrupt from the boundary: the file runs across it into unrelated data.
  • Correct by luck: the file happens to run into the right next fragment, and is indistinguishable from the other two without validation.

Stop and think

A carver's output opening in an image viewer proves only that the viewer is tolerant.

Reconstruct a fragmented JPEG

JPEG reconstruction is possible because the format is highly structured and its decoding is deterministic. You work at hex level, with a decoder, using these sources of evidence:

  • Filesystem allocation evidence. Even without the file's own record, the allocation bitmap, the runs of neighbouring files and the journal narrow the candidate clusters: a fragment cannot be in a cluster that another file has occupied continuously since before the image was written.
  • File signatures and markers. A JPEG begins with a start-of-image marker and ends with an end-of-image marker. Between them are marker segments, each with a two-byte code and a length: application segments (EXIF lives in one), quantisation tables, Huffman tables, the frame header (image dimensions and component layout) and the start-of-scan marker, after which entropy-coded scan data runs until the end marker. Every marker is a fixed point you can identify in a fragment.
  • Quantisation and Huffman tables. Scan data cannot be decoded without the tables from the header. Test a lone scan fragment against a candidate header by decoding: with the right tables it decodes cleanly, with the wrong ones it fails or produces noise. This is the principal method for matching headers to scans.
  • Scan data structure. Within the scan, any byte of value 0xFF is followed by 0x00 (byte stuffing) or by a marker. That property recognises a fragment of scan data, and a fragment that violates it is not JPEG scan data. If the encoder used restart intervals, restart markers appear at regular intervals with a cycling index: an ordering constraint (the indices must cycle in sequence) and a way to decode from the middle of a scan.
  • Structural decoding. Decode the candidate assembly and examine the result. A wrong boundary shows as a horizontal band where the content shifts, changes colour or becomes noise, because the decoder loses its place in the block sequence. A correct join decodes without error and the content is continuous. The number of blocks decoded before failure locates the boundary to within a few blocks, which converts to a byte range in the scan and hence to a cluster.
  • Neighbouring sectors and clusters. The next fragment is very often nearby: immediately after a gap occupied by another file, or the next free cluster at the time of writing. Bifragment gap carving tests candidate second fragments at increasing gaps from the first, validating each by decoding. For more than two fragments the search grows, and the allocation evidence above is what keeps it tractable.
  • EXIF metadata. Camera make and model, timestamps, dimensions and often the software that last saved the file. EXIF dimensions should match the frame header. Timestamps and camera identity link the file to other images on the system and to the filesystem timeline.
  • Embedded thumbnails. Most camera JPEGs carry a small, complete, independently decodable thumbnail of the same picture inside the EXIF segment. Comparing it with the reconstructed main image is a strong check on the assembly, and the thumbnail is often recoverable when the main image is not.
  • Candidate fragment comparison. Where several fragments could plausibly follow a given one, decode each candidate assembly and compare: image continuity across the join (content similarity of the rows either side), absence of decoder errors, restart marker sequence and any other sequence information the format carries, and agreement with the thumbnail and EXIF dimensions.

Keep the rejects

Record every candidate fragment considered and why each was accepted or rejected.

Assemble the toolkit

  • The carvers above (PhotoRec, Scalpel, Foremost, bulk_extractor) to locate headers, scan fragments, EXIF segments and thumbnails, and to produce the contiguous candidates that are the starting point.
  • X-Ways or an equivalent for hex-level examination with filesystem awareness, so a byte offset in the image can be translated to a cluster and back.
  • A hex editor with template support for inspecting JPEG marker structure by hand.
  • A JPEG decoder that reports errors and their positions rather than silently tolerating them, and a scriptable image library for decoding candidate assemblies in bulk.
  • Specialist fragmented-file carving tools, where available, that implement bifragment gap carving or structure-aware reassembly. Their output is subject to the same validation as any manual reconstruction; they do not remove the need for it.

Apply it to other formats

The same approach works with less internal structure to lean on:

FormatWhat helps
PNGA sequence of length-prefixed chunks, each with a CRC: strong validation at chunk boundaries.
Compressed archives and office documentsContainers with directories and checksums.
VideoA sequence of independently identifiable packets.
Plain text and raw dataAlmost no internal structure. Reassemble only on filesystem evidence and content plausibility, with correspondingly lower confidence.

Say what a reconstruction is and is not

  • It is a derived artefact, assembled by you from selected fragments in an order you chose.
  • It is not the original file, even if it is byte-identical to what the original was, because that identity cannot be observed; it can only be inferred from validation (section 20).
  • Opening, rendering and looking right means it is a hypothesis that has survived one weak test.
  • The report says how it was made, from which clusters, and how confident you are. It never describes the file as simply "recovered".

Example

Unallocated space contains a JPEG header with EXIF and a thumbnail at cluster 118,204, but the scan data terminates at the cluster boundary 12 clusters later without an end marker. Decoding the truncated scan produces the top third of an image consistent with the thumbnail. The allocation bitmap shows the next 40 clusters occupied by an intact document written before the image's EXIF timestamp, so the second fragment cannot be there. Candidate scan fragments (identified by byte-stuffing structure and by restart marker sequence continuing from index 3) are found at three locations within the following 2,000 clusters. Only the candidate at cluster 118,301 decodes without error through the join and continues the image content; it runs to an end marker 9 clusters later. The reconstruction, the two cluster ranges, the rejected candidates and the decoded comparison with the thumbnail are all recorded.

Before you move on

  • Surviving cluster runs used first, with intact and overwritten clusters recorded.
  • Carver output treated as truncated, corrupt or lucky until validated.
  • Every join validated by decoding, with thumbnail and EXIF checks where available.
  • All candidates, accepted and rejected, recorded with reasons.
  • Report describes the file as a reconstruction, with its clusters and confidence.