12. Hidden data

17. Analysis

Cross-platform data hiding

Whatever the platform, hidden data is found by looking for discrepancies: between what the filesystem says about a file and what the file contains; between the space the filesystem accounts for and the space the device has; between the activity the system records and the files that exist; between timestamps that should agree; and between how the system should behave and how it did. The methodology below is operating-system independent. The platform sections give the locations where each applies.

Discrepancy sources

• Filesystem metadata against file content: name, extension and recorded type against the actual bytes.
• Storage allocation against accounted space: partitions and volumes against the device size; allocated blocks against files; file sizes against extents.
• User activity against files present: a recent-items entry, a link file or a log line referring to a file that does not exist, or a mounted volume that is not on any disk.
• Timestamps against each other: filesystem times against internal metadata (EXIF, document properties), against journal entries, and against the ordering constraints the filesystem imposes.
• Expected behaviour against observed: a tool installed that has no legitimate purpose on that machine, a directory with unusual permissions, a service listening where none should.

File signature analysis

Compare each file's extension and the type the filesystem or application records with the magic value at the start of the file and its internal structure. A JPEG named as a spreadsheet, a ZIP archive named with an image extension, or a VeraCrypt container named as a video are found this way. Forensic suites and Autopsy report extension mismatches across a whole image; the file command does the same for individual files; and a hex viewer settles doubtful cases. A mismatch is an observed fact; the reason for it is interpretation, and legitimate causes (an application that saves with the wrong extension, a download that was renamed) must be considered before concealment is inferred.

Entropy analysis

Shannon entropy measures how evenly byte values are distributed. Encrypted, compressed and packed data all score near the maximum; text, most documents and executable code score lower and unevenly. Measured over a sliding window across an image, or file by file, entropy identifies candidates for: encrypted containers and volumes (uniform high entropy with no header), compressed archives (high entropy after a recognisable header), packed or obfuscated executables (high-entropy sections in a program), and containers embedded in other files (a high-entropy region inside a low-entropy file). Tools include binwalk -E, the ent utility, the entropy views in forensic suites, and short scripts. High entropy is an indicator, not proof: a file may be compressed rather than encrypted, random data may be exactly that, and a corrupt or trimmed region can score high. Confirm by looking for headers, by decompression, by the presence of tools and history, and by the recovery routes in section 12.

Deleted and residual data

Examine unallocated space, slack space, deleted directory entries and inodes, orphaned metadata records, filesystem journals and change logs, and residual structures from earlier filesystems or partition layouts, as set out in section 18. On each platform the sources differ (the change journal and shadow copies on Windows, the ext4 journal and orphan list on Linux, snapshots and FSEvents on macOS) but the question is the same: what existed that the current directory tree no longer shows?

Hidden storage

• Hidden partitions and unallocated gaps. Parse the partition table and account for every sector; mmls from The Sleuth Kit lists the unallocated ranges between and after partitions. A gap large enough to hold a filesystem, a partition whose type does not match its content, or a table entry that has been deleted but whose filesystem boot sector survives are all examined directly.
• Encrypted containers. Large files of uniform high entropy with no signature, especially with round sizes or names suggesting media. VeraCrypt hidden volumes occupy the free space of an outer volume and are undetectable by design; their existence is inferred, if at all, from the software's presence and from artefacts of use, and the report says "not excluded" rather than "not present".
• Loopback filesystems and virtual disks. Disk image files (raw, dmg, vhd, vhdx, vmdk, qcow2, vdi) are filesystems inside files. Find them by signature and by size, mount copies read-only, and examine them through the full process, including their own encryption and their own nested images.
• Nested filesystems and unusual layouts. A filesystem inside a file inside a volume inside an encrypted partition (section 14) is examined layer by layer. Unusual layouts, such as a second bootable partition, an oversized recovery partition or a filesystem starting at an unexpected offset, are listed and explained or reported as unexplained.

Metadata-based hiding

• NTFS alternate data streams attach additional named data to a file or directory; they are invisible to normal directory listings and can hold any content. On a live system dir /r and Get-Item -Stream * list them; forensic tools list them as separate entries against the parent file. The Zone.Identifier stream is legitimate and records download provenance; others are examined.
• Extended attributes on Linux and macOS, and resource forks on macOS, likewise hold arbitrary data invisible to a plain listing (sections 14 and 15).
• ACLs, unusual permissions and hidden attributes. Files readable by nobody, directories with the hidden and system attributes set on Windows, immutable flags on Linux, the hidden flag and the Finder "invisible" attribute on macOS, and permissions that differ from every neighbour are listed and examined.
• Filesystem-specific metadata can be abused: NTFS timestamps in the standard information attribute can be set by software, while those in the file name attribute normally cannot, so disagreement between them indicates manipulation; reparse points and junctions redirect paths; sparse and compressed attributes change how size relates to allocation. Each platform section lists its own.

File carving

Where filesystem metadata is damaged, missing or deliberately removed, carving recovers files by signature and structure from raw data: unallocated space, slack, a whole image, or a decrypted container. PhotoRec, Scalpel, Foremost and bulk_extractor and the carving modules of forensic suites do this at scale (section 19). Carving recovers content without names, paths or timestamps, produces many false and partial results, and, in its usual form, assumes each file is contiguous, which deliberately fragmented files and most large files on a busy volume are not. Carved output is tool-reported information to be validated (section 20), not recovered evidence.

Steganography

Steganography hides data inside a carrier: the least significant bits of image pixels or audio samples, unused fields and frames in video, the formatting, revision history or embedded objects of documents, and the comment fields, padding or appended data of archives. Detection is driven by indicators, not applied blindly to every file: the presence of steganography software or its traces (installed programs, recent files, command history, browser downloads), carrier files that are near-duplicates of others but differ in size or in their low-order bits, image files larger than their dimensions and format warrant, files with data appended after the format's end marker, and archives whose stored sizes do not add up. Given an indicator, examine the specific carriers: statistical tests for LSB embedding, comparison against the original where one exists, structural parsing to find appended or embedded data (binwalk finds embedded files by signature; strings exposes embedded text; ExifTool exposes metadata fields, including unusual or oversized ones), and, where the tool is identified, extraction with it. A positive statistical result is an indicator; extraction of coherent content is evidence.

Fragmented files

Where a file has been deliberately fragmented to defeat carving: inspect the filesystem's allocation metadata for a surviving record and reconstruct the extent or cluster chain from it where possible; where the record is gone, correlate candidate fragments by file structure, headers and footers, internal markers, embedded metadata, sequence information (restart markers, chunk indices, packet numbers) and content similarity across the join; reconstruct manually where automated recovery fails; and retain the reconstruction process and the provenance of every fragment. Section 19 gives the full method and section 20 the validation.

Tools and their purposes

• Autopsy and The Sleuth Kit: filesystem parsing, deleted and orphaned entries, extension mismatch detection, partition layout, unallocated space extraction.
• bulk_extractor: scanning raw data for features (email addresses, URLs, EXIF, embedded files) without regard to filesystem structure.
• PhotoRec, Scalpel and Foremost: signature-based carving.
• ExifTool: reading and comparing embedded metadata across image, audio, video and document formats.
• binwalk: signature scanning for embedded files and firmware, and entropy plotting.
• strings: extracting printable text from binary data, for a first look at unknown files and memory.
• Entropy tools (ent, binwalk, suite views, scripts): locating encrypted, compressed and packed regions.
• Native operating system utilities (file, dir /r, xattr, getfattr, lsblk, diskutil): enumerating attributes and layout, on the live system with a recorded footprint or against a mounted copy.
• Validated commercial suites (X-Ways, EnCase, FTK, Magnet and others): integrated parsing, carving, indexing and reporting with documented validation.
• Memory analysis frameworks (Volatility and equivalents): processes, keys, credentials and mounted-volume state from memory images (section 21).
• ddrescue and equivalents: fault-tolerant imaging of failing media with a map of unread regions (section 8).

No single tool is authoritative. Where a finding matters, confirm it with a second tool or by direct inspection of the bytes.

Example. A directory of holiday photographs contains one JPEG twice the size its dimensions warrant. ExifTool shows ordinary camera metadata. A structural parse finds the end-of-image marker two fifths of the way through the file, followed by data with a 7-Zip signature. The appended archive is extracted from a copy, found to be password-protected, and its entropy and header recorded. The report states the observed facts (sizes, offsets, signatures), the tool-reported information (the parser's marker positions), the interpretation (an archive appended to an image) and the limitation (archive not opened, contents unknown).