The Depths of Windows I/O (Part 5) — NTFS Internals: Understanding the File System Through the MFT

Across the four parts so far, we have followed how an I/O request flows (Parts 1–3) and how the cache manager receives it (Part 4). In the end, the request arrives at the file system. This time it’s the turn of its leading example, NTFS.

The perspective shifts here. Until now the story has been dynamic — the flow of a request. This time it’s a static story about how data is laid out on disk. The true identity of the invisible “Zone.Identifier” attached to a downloaded file. Why copying ten thousand small files is so much slower than copying one file of the same total size. How far it’s really true that “NTFS is journaling, so it’s safe” — all of it can be explained from this structure.

This is Part 5 of the series “The Depths of Windows I/O”.

Prerequisite for this part: it helps to already have a grip on the basics of IRPs and the device stack from Part 1. That said, so this article stands on its own, here are definitions of the terms from earlier parts that appear in the body below.

Table 1: Terms from earlier parts assumed in this instalment

One more thing: the two stages covered in Part 1 — cleanup (when the last handle closes) and close (when every reference inside the kernel has gone) — are also used in the explanation of file deletion in Section 4.

1. The Bottom Line First

• NTFS is built around the MFT (Master File Table). Every file is tracked as a record in the MFT’s ledger, and everything about a file lives either “inside the MFT entry” or “in the area outside the MFT that the entry points to” (Section 2).¹
• A file’s substance is “a collection of attributes.” A small file fits entirely inside its MFT record, data and all (resident); a large file holds only a reference to a run of clusters (non-resident). This is what explains why processing huge numbers of small files is slow (Section 2).¹
• A file can hold more than one data stream (multiple data streams). The data you normally see is the “unnamed stream”; file.txt:name lets a file carry additional streams alongside it. This is the true identity of Zone.Identifier (Mark of the Web) (Section 3).²
• Names are attributes too. Attaching more than one name to the same record is a hard link. The 8.3 short name also lives alongside it as “one more name” (Section 4).³⁴
• Reparse points are the official mechanism for “open here, end up somewhere else.” Symbolic links, junctions, and OneDrive Files On-Demand are all applications of this same tagged data (Section 5).⁵⁶
• There are two journals. $LogFile is for recovering metadata consistency (a write-ahead log so the volume doesn’t break), while the USN journal is for recording change history (a ledger of what changed). Their roles are completely different (Section 6).⁷⁸
• “Size” and “size on disk” are two different things. Sparse files and compression are what drive them apart. The background to “a compressed file never becomes asynchronous,” which we saw in Part 2, lives here too (Section 7).⁹¹⁰

2. Everything Is a Record in the MFT

2.1. The Volume’s Ledger

Formatting an NTFS volume creates the MFT (master file table) along with a set of metadata files whose names begin with $. The MFT holds at least one entry for every file on the volume, and that includes an entry for the MFT itself.¹

flowchart TB
    subgraph VOL["NTFS volume"]
        MFT["$MFT — the master file table<br/>Ledger of every file's record, itself included"]
        LOG["$LogFile — transaction log for metadata operations, Section 6"]
        BITMAP["$Bitmap — cluster usage status"]
        OTH["$Boot / $Secure / $UpCase, and other<br/>metadata files"]
        DATA["User data area<br/>Where non-resident data lives"]
    end
    MFT -->|"records point to the location"| DATA

Figure 1: The structure of an NTFS volume. NTFS’s design principle is that even the file system’s own management information is held as a file

Information about a file — its size, timestamps, access permissions, and even the content of the data itself — is either stored inside the MFT entry, or stored in an area outside the MFT whose location the MFT entry describes.¹ Deleting a file marks its entry as “free” for reuse, but the MFT itself never shrinks. The official documentation even goes as far as describing the MFT’s life cycle: an area called the MFT zone is reserved to keep the MFT contiguous, and once the volume starts filling up, MFT fragmentation begins.¹

2.2. A File Is a Collection of Attributes: Resident and Non-Resident

The contents of a file record are a list of attributes: standard information (timestamps and so on), the file name, security, and data. There’s an important fork in the road here.

flowchart TB
    subgraph REC["MFT file record — the ledger entry for one file"]
        STD["Standard information attribute<br/>Timestamps and attribute flags"]
        FN["File name attribute<br/>Can hold more than one - Section 4"]
        DATA["Data attribute"]
    end
    Q{"Is the data small?"}
    RES["Resident<br/>The data itself fits inside the record<br/>Reading it needs only an MFT access"]
    NONRES["Non-resident<br/>The record holds only a reference to a run of clusters<br/>The real data sits in the user data area"]
    DATA --> Q
    Q -->|"up to roughly a few hundred bytes"| RES
    Q -->|"beyond that"| NONRES

Figure 2: A file record is a collection of attributes. If the data is small, it “resides” inside the record

It helps to lay out, side by side, how the contents of the same record change between resident and non-resident.

flowchart LR
    subgraph RES2["Resident - a small file"]
        RA["MFT file record, fixed length<br/>Standard information / File name / Security<br/>─────────────<br/>Data attribute = the content itself<br/>e.g. 'setting=1' goes straight in here"]
        RB["There's no separate location on disk<br/>Reading it is complete with just an MFT access"]
        RA --> RB
    end
    subgraph NON2["Non-resident - a large file"]
        NA["MFT file record, fixed length<br/>Standard information / File name / Security<br/>─────────────<br/>Data attribute = a table of data runs<br/>A list of 'starting where, how many clusters'"]
        NB["User data area<br/>Run 1 - contiguous clusters"]
        NC["User data area<br/>Run 2 - contiguous clusters elsewhere"]
        NA -->|"points to the location"| NB
        NA -->|"points to the location"| NC
    end

Figure 3: Resident versus non-resident, side by side. For a non-resident file, all the record holds is a table (the data runs) of where the real data is and how much of it there is

The more data runs there are, the more scattered regions you have to hop across to read a single file. That’s the true identity of fragmentation, which we come to next.

This structure explains several phenomena you run into in the field.

• Why copying ten thousand small files is slow. Every single file triggers metadata operations — creating an MFT record, registering a name, setting security — and the bookkeeping ends up dominating over the actual data transfer. (And each one of those operations is also something for the filters we’ll meet in Part 6 to inspect.)
• The true identity of fragmentation. Non-resident data is recorded as “a series of contiguous ranges of clusters (runs).” When a contiguous region can’t be found, the number of runs grows, and so does the number of seeks a read needs — that’s fragmentation. You can peek at the actual layout of runs with fsutil file layout.
• “Folders” aren’t special either. A directory is “a file that holds an index from file names to MFT record numbers.” On the ledger, everything sits on the same underlying mechanism.

3. Data Is Just One of the “Streams”

3.1. One File, Multiple Byte Sequences

In NTFS, a single file can hold more than one data stream. What you normally read and write with ReadFile/WriteFile is the unnamed default stream; the syntax filename:streamname lets you create an alternate data stream (ADS).²

flowchart LR
    subgraph F["A file named report.docx - a single MFT record"]
        D0["Default stream, unnamed<br/>= the content you normally see"]
        D1[":Zone.Identifier<br/>Provenance info, Mark of the Web"]
        D2[":any name<br/>Extra data specific to an application"]
    end

Figure 4: Multiple data streams. Only the default stream shows up in Explorer’s size display

The most familiar ADS is Zone.Identifier. For a file downloaded through a browser, this stream records its provenance (that it came from the internet, for example), and it’s what SmartScreen’s “Windows protected your PC” and Office’s Protected View use as evidence. We covered the front side of this mechanism in “Why Windows Shows "Windows protected your PC"” — the identity behind the scenes turns out to be nothing more than an NTFS stream.

3.2. Pitfalls Developers Run Into

• It’s invisible. It doesn’t show up in Explorer’s size or in a dir listing. You can check for it with dir /r or Sysinternals’ streams.¹¹
• It doesn’t travel. Because ADS is an NTFS feature, it tends to get lost when you copy to a FAT USB drive or via cloud storage. This is why “the download warning disappeared after I copied the file” happens.
• Your own application can open one too. Simply including a colon in the path, as in CreateFile("data.txt:meta", ...), lets you read and write it.² It’s convenient, but you inherit the “doesn’t travel” property from the previous point along with it, so it’s not the place to put the substance of business data.

4. Names Are Attributes Too — Hard Links and 8.3 Names

4.1. Hard Links — Multiple Names for the Same Record

In Figure 2 we noted that “a file can hold more than one file name attribute.” Multiple paths within the same volume referencing a single file — that’s a hard link (CreateHardLink / mklink /H).³

flowchart TB
    subgraph DIR1["The index for C:\app\"]
        E1["config.json points to record #1234"]
    end
    subgraph DIR2["The index for C:\backup\"]
        E2["config-link.json points to record #1234"]
    end
    REC["MFT record #1234<br/>The data itself, or a reference to its runs<br/>Link count 2"]
    E1 --> REC
    E2 --> REC

Figure 5: A hard link. Both directory indexes simply point to the same MFT record, and each is equally “the real thing”

Because it’s the same file regardless of which name you change it through, the content matches immediately.³ And this changes what “deletion” means: DeleteFile means “detach one name,” and the entity itself disappears only once the last name has been detached, every open handle has closed, and every reference inside the kernel — such as a memory-mapped section — has also gone. The two stages we saw in Part 1, cleanup (the last handle) and close (the last reference), turn out to govern the lifespan of deletion in exactly the same way. Note too that attribute display has its quirks: the official documentation notes that changing an attribute through one link can leave the apparent display through another link stale.³

4.2. The 8.3 Name — Another Hidden Name

For historical compatibility, NTFS can automatically generate an 8.3-format short name, such as REPORT~1.DOC, for a long file name. This too lives alongside the record as “one more name.” In a folder holding huge numbers of files, generating short names and avoiding collisions becomes a real cost, so fsutil 8dot3name lets you disable generation or strip existing short names (a practical point worth noting: because an old application that records a registry path by its short name can break if you strip it, there’s an inspection feature you can run before stripping).⁴

What’s worth flagging here is that whether a short name exists at all depends on the environment. The default behaviour is governed by the registry value NtfsDisable8dot3NameCreation, which has four settings: 0 (generate on every volume), 1 (never generate on any volume), 2 (set per volume), and 3 (never generate outside the system volume).⁴ Since choosing 2 lets you flip the setting per volume, it is not necessarily true that “on Windows, a short name like PROGRA~1 always exists.” Before writing code or a procedure that depends on short names, check the current state with fsutil 8dot3name query C: (omit the volume to see the default setting shared across all volumes).

The pitfalls surrounding paths and names (MAX_PATH, reserved names, trailing dots) are covered in detail in “MAX_PATH and Windows Path/Filename Pitfalls — the 260-Character Limit, Reserved Names, Trailing Dots, and Case Sensitivity.” Put together with name resolution (the object manager) from Part 1, this section (names inside the file system) completes the full picture of “names” in Windows.

5. Reparse Points — The Mechanism Behind “Open Here, End Up Somewhere Else”

Files and directories can carry a reparse point. Underneath, it’s an attribute consisting of “a tag plus user-defined data.” When the file system opens a file that carries a reparse point, processing gets hijacked according to the tag — either a filter driver that understands the tag takes over processing, or, for a name-redirection-style tag, resolution starts over using the target path.⁵

sequenceDiagram
    participant App as Application
    participant IOM as I/O manager
    participant FS as NTFS

    App->>IOM: CreateFile("C:\data\link.txt")
    IOM->>FS: IRP_MJ_CREATE (the world of Part 1)
    Note over FS: Finds a reparse point on the target<br/>returns the tag and its data
    alt Symbolic link / junction (name redirection)
        FS-->>IOM: "The real location is here"
        IOM->>FS: Restart resolution using the target path
    else A filter-managed tag (cloud files, etc.)
        Note over FS: A filter that understands the tag<br/>takes over processing (Part 6)
    end

Figure 6: Resolving a reparse point. It’s the official hook that intercepts the “open” operation

A row of familiar features all sit on top of this single mechanism.

• Symbolic links (mklink) — a signpost holding the target path. It can also point to a different volume or a UNC path.⁶
• Junctions / mount points — a long-standing mechanism that connects a directory to a location on a different local volume.³
• OneDrive Files On-Demand — represents a file whose actual data isn’t present locally with a reparse point, and the instant it is opened, a filter downloads it and hands over the content. This is the true identity of “it’s visible in Explorer, but opening it kicks off network traffic.” (The filter mechanism itself is covered in Part 6.)

There is one practical caution: the far end of a path is not necessarily really that local location. A tool that walks a tree recursively can loop through a junction; size totals can double-count; a backup can trigger mass hydration of cloud files — code that doesn’t know reparse points exist steps straight into these. Checking the FILE_ATTRIBUTE_REPARSE_POINT attribute from FindFirstFile and its relatives is the starting point for guarding against them.⁵

6. Two Journals — $LogFile and the USN

People often say “NTFS is a journaling file system,” but NTFS actually has two journals with different roles. Conflate them and you’ll misread what they actually guarantee.

flowchart TB
    subgraph J1["$LogFile - a write-ahead log, so the volume doesn't break"]
        A1["Records metadata operations, record updates, renames, etc.<br/>to the log before they execute"]
        A2["At the next startup after a system failure,<br/>replays the log to restore structural consistency"]
        A1 --> A2
    end
    subgraph J2["USN journal - change history, so you can learn what changed"]
        B1["Every time a file or directory changes,<br/>records the nature of the change and its name"]
        B2["Lets backup, search indexing, and sync tools<br/>learn 'what changed since last time' without a full scan"]
        B1 --> B2
    end

Figure 7: Two journals. $LogFile exists so nothing breaks; the USN exists so you can learn what changed

Laid out as a table, the difference looks like this.

Table 2: The two journals compared

• $LogFile (the transaction log) is the write-ahead log of metadata operations. Even after a system failure, NTFS uses this log together with checkpoint information at the next startup to automatically recover the file system’s consistency.⁷ What’s protected here is the structure. As we saw in Part 4, dirty data content sitting in the cache can still be lost to a power failure — the correct reading is “the volume won’t break, but the last write can still vanish.”
• The USN journal (change journal) is a ledger that records the nature of the change and the name of the target every time a file or directory within the volume changes.⁸ It’s the mechanism that lets backup tools and indexers pick up “only what changed since last time” without a full scan, and it’s also used to avoid rebuilding an index after a failure.⁸ In practice, it’s worth remembering that you can use fsutil usn readjournal for investigation as a reconciliation ledger to make up for what FileSystemWatcher misses (see “A Practical Guide to FileSystemWatcher — Handling Missed and Duplicate Events”).

7. Sparse Files and Compression — the Story of Two “Sizes”

In NTFS, a file’s logical length and the region actually allocated to it are managed separately — that’s “Size” and “Size on disk” in the Properties dialog. Two things are chiefly responsible for pulling them apart.

A sparse file doesn’t allocate real storage to ranges that consist of nothing but zeros; it manages them as “holes.”⁹ It’s entirely normal for a virtual disk file with a logical size of 42 GB to be using only 500 MB on disk. Reading a hole returns zeros, and writing to one allocates just as much space as you wrote.

flowchart LR
    subgraph L["The logical file - size 1GB"]
        R1["Data 10MB"]
        H1["Hole, zeros - 500MB"]
        R2["Data 5MB"]
        H2["Hole, zeros - the rest"]
    end
    subgraph P["The allocation on disk - 15MB plus bookkeeping"]
        A1["Run - the substance of R1"]
        A2["Run - the substance of R2"]
    end
    R1 --> A1
    R2 --> A2

Figure 8: A sparse file. A “hole” has no allocation, which is what drives the logical size and the size on disk apart

NTFS compression compresses and stores data in units called compression units.¹⁰ It’s transparent and convenient, but the cost isn’t transparent — decompression and recompression run on every read and write, and fragmentation tends to progress faster too. And as we saw in Part 2, Section 5, access to a compressed file never becomes asynchronous (the file system converts it to synchronous). This is one of the places to suspect when “some files don’t get faster even though I switched to asynchronous I/O.”

The actual, allocation-based size can be retrieved with GetCompressedFileSize. When investigating a mismatch between “total file size” and “disk usage,” the standard playbook is to check, in order, sparseness, compression, ADS (Section 3), and cluster rounding-up — four suspects in all.

8. See It for Yourself

Once again, everything here can be observed on your own Windows machine (some of it needs administrator rights). So you can judge for yourself whether the results are correct, each command comes with a note on where to look and what it tells you.

:: View alternate data streams
dir /r C:\Users\%USERNAME%\Downloads

Where to look: below the normal file line, an indented line in the form filename:Zone.Identifier:$DATA appears with a length next to it. If that line is there, the file has Mark of the Web attached to it (Section 3). Files downloaded through a browser have it; files you created yourself don’t. Run this in both places and compare, and whether the ADS is present becomes obvious.

:: View a file's layout (runs) and attributes on the MFT
fsutil file layout C:\path\to\file.dat

:: View just the extents (a subcommand documented in the official docs)
fsutil file queryextents C:\path\to\file.dat

Where to look: for each stream, layout lists the size, the allocated size, and — if non-resident — the list of extents (each a triple of VCN, LCN, and cluster count). A very small file that shows no extent lines is resident (Section 2.2); if it’s split across multiple lines, it’s fragmented. Running this against a text file of a few bytes and a file of a few hundred MB and comparing the output is the quickest way to get a feel for resident versus non-resident.

:: The 8.3 short-name generation setting, and any existing short names
fsutil 8dot3name query C:
dir /x

Where to look: query returns whether short-name generation is enabled or disabled for that volume (omit the volume to see the default setting shared across all volumes).⁴ dir /x shows a short-name column next to the long name, so an empty column means no short name was generated. You can confirm on your own machine what Section 4.2 called “you can’t assume a short name exists.”

:: The state of the USN journal
fsutil usn queryjournal C:

Where to look: this displays the journal ID, the valid range of USNs (First USN / Next USN), the target maximum size (MaximumSize), and the allocation unit (AllocationDelta).¹² Create some file and run it again, and Next USN should have advanced — confirming that “changes are being recorded.” MaximumSize is a rough guide to “how far back you can reach,” which we touched on in Section 6. On a volume where the journal is disabled, this returns an error.

:: Checking reparse points (the target and the tag)
dir /aL C:\Users\%USERNAME%
fsutil reparsepoint query "C:\Users\%USERNAME%\OneDrive"

Where to look: whatever dir /aL lists is a reparse point (one with FILE_ATTRIBUTE_REPARSE_POINT set). Symbolic links and junctions show up tagged with a type such as <SYMLINKD> or <JUNCTION>. fsutil reparsepoint query displays the reparse tag’s value and, for a name-redirection-style tag, the target path. Pointing it at something that isn’t a reparse point returns an error, so getting an error is itself confirmation that “this is just an ordinary folder.”

Trace file operations with Procmon, and today’s cast of characters flows past under their real names (writes to $LogFile, paths with a stream name attached, reparse processing). See “A Practical Guide to Process Monitor (ProcMon) — Pinpointing "Settings Not Applied" and "ACCESS DENIED" in 10 Minutes” for how to use it.

9. Summary

• NTFS is built around the MFT. Every file is a record in the ledger, and information sits either “inside the record” or “in the external area the record points to.” Small data is resident, large data is referenced by runs, and the slowness of processing huge numbers of small files, along with fragmentation, are both consequences of this structure.¹
• A file can hold more than one data stream. Zone.Identifier (Mark of the Web) is nothing but an ADS: it’s visible with dir /r, and it doesn’t travel outside NTFS.²¹¹
• Names are attributes, and a file can have more than one. A hard link is an equal-status name pointing to the same record; the 8.3 name is one more name kept for compatibility. “Deletion” means “detaching a name,” and the entity itself disappears only once the last name, the last handle, and every reference inside the kernel (a mapped section and so on) are all gone.³⁴
• Reparse points are the official hook into “opening”, and symbolic links, junctions, and Files On-Demand are all applications of it. Code that walks a tree needs to be aware of FILE_ATTRIBUTE_REPARSE_POINT.⁵⁶
• There are two journals. $LogFile is for structural-consistency recovery (so nothing breaks); the USN is change history (what changed). “It’s journaling, so the data is safe too” is not correct — data durability is something you build with the tools from Part 4.⁷⁸
• Logical size and allocation are separate things. Sparseness, compression, ADS, and cluster rounding-up are the four big causes of “the sizes don’t match.” Together with the fact that compressed files never go asynchronous, this is a set of tools worth keeping in your back pocket for performance investigations.⁹¹⁰

The series concludes next time, in Part 6, “Filter Drivers and Minifilters — Why Procmon and Antivirus Scanners Can Intercept I/O.” How do the “characters standing in between” who have made occasional appearances since Part 1 — antivirus, Procmon, OneDrive, encryption — actually intercept I/O? As the finale of the series, we’ll reveal the true identity of the residents standing in the gaps of the device stack.

Related Articles

• The Depths of Windows I/O (Part 1) — Every Read and Write Becomes an IRP: The Big Picture of the I/O System
• The Depths of Windows I/O (Part 2) — Synchronous and Asynchronous I/O: What OVERLAPPED Really Means
• The Depths of Windows I/O (Part 4) — Cache Manager: When Does Your WriteFile Actually Reach the Disk?
• Why Windows Shows “Windows protected your PC”
• MAX_PATH and Windows Path/Filename Pitfalls — the 260-Character Limit, Reserved Names, Trailing Dots, and Case Sensitivity
• A Practical Guide to FileSystemWatcher - Handling Missed and Duplicate Events
• Pitfalls of Network Drives and UNC Paths — Working With File Servers (Shared Folders) From a Business Application
• A Practical Guide to Process Monitor (ProcMon) — Pinpointing “Settings Not Applied” and “ACCESS DENIED” in 10 Minutes

Related Consulting Areas

KomuraSoft LLC handles the design and investigation of Windows business applications rooted in how NTFS actually works — puzzling behaviour around file size and copy performance, bugs tangled up with links and streams, and more.

• Windows Application Development
• Bug Investigation & Root Cause Analysis
• Legacy Asset Utilisation & Migration Support
• Contact Us

References

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Frequently Asked Questions

Common questions about the topic of this article.

What is the MFT (Master File Table)?

It's the data structure at the heart of an NTFS volume — a ledger that holds at least one entry (a file record) for every file on the volume, including an entry for the MFT itself. Everything about a file, from its size, timestamps, and access permissions down to the content of the data itself, is stored either inside an MFT entry or in an area outside the MFT that the entry points to. A small file fits entirely, data and all, inside its MFT entry (resident); a large file has only a reference to the layout of its data (a run of clusters) recorded in the entry (non-resident). Deleting a file marks its entry as free for reuse, but the size of the MFT itself never shrinks.

What is the invisible "Zone.Identifier" data attached to files?

It's one of NTFS's multiple data streams (alternate data streams). In NTFS, a single file can hold several byte sequences (streams); what you normally read and write is the unnamed default stream. Windows records where a file came from — that it was downloaded from the internet, for example — in an additional stream you specify with a colon, as in "file.txt:Zone.Identifier". This is the so-called "Mark of the Web," and it's what SmartScreen and Office's Protected View use as evidence in their judgments. Alternate streams don't appear in Explorer's size display; you can check for them with the dir /r command or Sysinternals' streams tool. It's also worth noting that they aren't preserved when copied to a non-NTFS file system such as FAT.

What's the difference between a hard link and a symbolic link?

A hard link is "one more name of equal status added, pointing at the same file entity — the same MFT record." It can only be created within the same volume, accessing the file through any of its names gives you the same file, and deleting one name doesn't remove the file as long as another name still remains. A symbolic link is "a signpost that directs you to a different path," implemented as a reparse point. Since it only holds the target path as a string, it can point to a different volume or even a remote location, but it becomes a dead end if the target disappears. In practice, the rule of thumb is: use a hard link to share the underlying entity (which changes what deletion means), and use a symbolic link to redirect a path (for relocation or redirection).

NTFS is a journaling file system, so does that mean data survives a power failure?

You need to understand exactly what's protected. What NTFS's transaction log ($LogFile) protects is the consistency of the file system's structure (its metadata). Even if a system failure occurs, NTFS automatically restores consistency using the log the next time it starts up, preventing the situation where the volume is corrupted and unreadable. But that doesn't mean the actual content of a file that was mid-write gets restored. As we saw in Part 4 of this series, dirty data sitting in the cache is lost in a power failure. So the correct understanding is "the volume won't break, but the content of the last write can still vanish" — if you need durability for the data itself, you have to build it in yourself, with FlushFileBuffers, WRITE_THROUGH, or an application-level write design such as write-to-temp-file-then-rename.

Why does a file's "size" differ from its "size on disk"?

Because NTFS manages a file's logical length and the disk space actually allocated to it separately. Even an ordinary file shows some difference because allocation is rounded up to whole clusters (4KB by default), but sparse files and compressed files are where the gap becomes large. A sparse file doesn't allocate real storage to ranges that run on as zeros — it manages them as "holes" — so it's entirely possible for a file with a logical size of several gigabytes to use only a few megabytes on disk. A compressed file has only its post-compression size allocated. Conversely, if the "size on disk" looks larger, cluster rounding or an alternate data stream is often the cause.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.