The Depths of Windows I/O (Part 4) — Cache Manager: When Does Your WriteFile Actually Reach the Disk?

WriteFile returned success. So where is the data right now?

The answer is, almost certainly, still not on disk. It has only been copied into an in-memory cache. That’s why “I saved it, but after the power cut it was gone” happens; that’s why file-copy benchmarks post physically impossible speeds; and that’s why databases dutifully call fsync.

Part 4 of the series “The Depths of Windows I/O” is about the thing that sits in the middle of all this: the cache manager. In Part 2 we noted that “if the data is already in the cache, even asynchronous I/O completes synchronously,” and in Part 1 we left “the shortcut that skips building an IRP — fast I/O” as homework. This instalment collects on both of those threads.

1. The Bottom Line First

• Windows’ file cache is write-back. Reads are served from the system file cache first, and writes go there first too. Getting the data onto disk is something the OS does afterwards.¹
• The cache is, at bottom, a file mapping. The cache manager maps 256KB sections of a file into slots in the system’s address space, and cache-enabled reads and writes become memory copies to and from that view (Section 2).¹
• Writes are caught up by a lazy writer that runs every second. An application crash does not lose data, but a power loss or an OS crash loses whatever dirty cache had not yet been written back (Section 4).¹
• There are three tools for “definitely written.” FlushFileBuffers (equivalent to .NET’s Flush(true)), FILE_FLAG_WRITE_THROUGH, and FILE_FLAG_NO_BUFFERING. Flushing on every single write is inefficient, and the official documentation recommends combining NO_BUFFERING with WRITE_THROUGH instead (Section 5).²¹
• NO_BUFFERING comes with alignment requirements. Size and offset must be exact multiples of the sector size, and the buffer address must be aligned to a physical sector boundary too. And even under NO_BUFFERING, metadata continues to be cached (Section 5.3).³¹
• A mapped view and the cache share the same data. Memory-mapped files and ordinary cached I/O stay coherent, and persisting a mapping takes two steps: FlushViewOfFile followed by FlushFileBuffers (Section 6).⁴⁵
• Synchronous reads and writes hitting the cache may not even generate an IRP. The fast-I/O shortcut goes straight into the cache manager — the answer to Part 1’s homework (Section 7).⁶

2. What the Cache Really Is — Files Mapped into Memory

2.1. 256KB Slots and Memory Copies

If you picture Windows’ file cache as “a container of disk blocks,” a lot of its behaviour stops making sense. The correct picture is this: the cache manager maps 256KB sections of a file into “slots” in the system address space, and cache-enabled reads and writes are carried out as memory copies between that slot and the application’s buffer.¹

flowchart TB
    subgraph U["App - user mode"]
        BUF["App buffer<br/>the region passed to ReadFile/WriteFile"]
    end
    subgraph S["System address space"]
        SLOT["System file cache<br/>slot mapping a 256KB section of the file"]
    end
    DISK[("File on disk")]
    BUF <-->|"ReadFile/WriteFile =<br/>memory copy to and from the slot"| SLOT
    SLOT <-->|"Initial-access reads and<br/>later write-backs happen page by page"| DISK

Figure 1: The reality of cache-enabled I/O. What an app sees as “reading or writing a file” is, most of the time, just a memory copy

One point that’s easy to get wrong: 256KB is the granularity of the view (the mapping), not a claim that disk I/O always happens in 256KB units. Pages inside a slot are read in as needed, and the actual amount of disk I/O varies with the request size and the access pattern. Reading a section for the first time triggers disk I/O to fill it (this is where an IRP from Part 1 travels down into the lower storage stack). If it’s already in the cache, the read completes as a pure copy. What we saw in Part 2, Section 5 — “a cache hit completes synchronously even when issued asynchronously” — is exactly this “answer on the spot if you can” behaviour showing itself. Conversely, when a page is still cache-enabled but not actually resident in memory, there’s no asynchronous mechanism for page-fault handling, so an asynchronous read can end up being processed synchronously — a trap we also covered in Part 2.⁷

2.2. What’s Really Behind “Free Memory Went Down”

Whether caching and read-ahead are used is managed per open (per file object)¹, but the cached data itself is shared per file (per stream). Opening the same file multiple times doesn’t produce separate caches — every handle sees the same cached content (this is the foundation for the coherency covered in Section 6). The cache stays under the cache manager’s control for as long as Windows is running.¹ Copy a large file, or read and write heavily, and free physical memory keeps getting turned into cache. Even when Task Manager’s free memory looks like it’s shrinking, most of that is “standby memory being put to good use, ready to be handed back the moment an application asks for it.”

You can confirm this on screen. Open Task Manager > Performance > Memory, and the “Memory composition” bar at the bottom is divided into In use / Modified / Standby / Free. Most of the file cache lands in Standby, and is totalled as “Cached” in the panel on the right. For a finer-grained view, Resource Monitor > Memory tab shows the same breakdown with figures attached. Copy a multi-gigabyte file and then look again: standby goes up and free goes down, while “in use” barely moves — visible proof that this isn’t “memory got eaten up” but “free memory got put to work as cache.” We also covered this distinction in practice, for diagnosing memory shortages, in “Telling a .NET GC Wait Apart From a Memory Leak.”

3. Read-Ahead — Speculating on Reads

Based on past access patterns, the cache manager preloads the section it expects to be read next (read-ahead). For a file being read in order, the next chunk of data is often already sitting in the cache before the application even asks for it — that’s the trick behind how fast sequential reads feel. The amount read ahead isn’t fixed; it varies with the pattern detected and the request size.

flowchart LR
    A["History of the app's read requests<br/>reading sequentially from the start"]
    D{"Cache manager detects<br/>the pattern"}
    R["Read-ahead - loads the next section<br/>before it is requested<br/>amount varies with pattern and request size"]
    H1["Hint FILE_FLAG_SEQUENTIAL_SCAN<br/>= read ahead aggressively"]
    H2["Hint FILE_FLAG_RANDOM_ACCESS<br/>= suppress read-ahead, since it would be wasted"]
    A --> D
    D --> R
    H1 -.-> D
    H2 -.-> D

Figure 2: Read-ahead. On top of detecting the access pattern, flags on CreateFile can supply a hint

The FileOptions.SequentialScan / RandomAccess values from Part 1’s mapping table are hints to this read-ahead engine. The former suits batch processing that scans “everything, in order”; the latter suits access that hops around, such as walking an index — think of them as flags that tell the OS about a future only the application knows about, and their purpose becomes clear.

4. Lazy Writing — What WriteFile’s “Success” Actually Means

4.1. The Lazy Writer Shows Up Every Second

The write side is a write-back cache. WriteFile returns success as soon as the data has been copied into a slot, and getting it onto disk is deferred. This policy of “write later” is lazy writing.¹

The thing that carries out the write-back is the lazy writer, which the cache manager kicks off once every second. It queues up one-eighth of the pages that haven’t been flushed recently, and queues more on top of that if there’s a lot to write. Note that temporary files created with the FILE_ATTRIBUTE_TEMPORARY attribute are excluded from the lazy writer’s flush target — there’s no point writing out something that’s expected to be deleted shortly anyway.¹ This is only a hint conveyed by the attribute, though: the pages can still be written back under memory pressure, and it does nothing for a file that merely has a name that “looks temporary.”

sequenceDiagram
    participant App as App
    participant C as System cache
    participant LW as Lazy writer, runs every second
    participant D as Disk

    App->>C: WriteFile(data)
    Note over C: Copies into the slot and<br/>marks the page dirty, not yet written
    C-->>App: TRUE is returned right away
    Note over App,C: From here until the write-back is the "dangerous window"<br/>a power loss or OS crash loses this data
    LW->>C: Picks 1/8 of the dirty pages
    LW->>D: Writes them back in a batch
    Note over D: Only now is it actually persisted

Figure 3: Lazy writing. WriteFile succeeding means “handed off to the OS,” not “made durable”

4.2. What Happens, and How Much Is Lost

Let’s be precise about what that “dangerous window” means. The outcome depends on the kind of failure.

flowchart TB
    W["Data right after WriteFile succeeds<br/>a dirty page in the cache"]
    Q{"What happened"}
    A1["The app process<br/>crashed or was killed"]
    A2["The whole OS went down<br/>power loss, blue screen"]
    S["The data survives<br/>the cache belongs to the OS, so<br/>the lazy writer writes it back as scheduled"]
    L["Dirty pages are lost<br/>only what already reached disk survives"]
    W --> Q
    Q --> A1
    Q --> A2
    A1 --> S
    A2 --> L

Figure 4: The dividing line between failure types. The cache belongs to the OS, not to the process

• The data does not disappear just because the app dies. Once the data has been copied into the cache, the OS owns it. This is why “the app crashed right after saving, but the file was fine” holds true.
• If the OS itself dies, the dirty portion is lost. How often flushing runs is tuned as a trade-off between performance and reliability, and the documentation states plainly that “if a sudden loss of power occurs, cached data is lost.”¹

Which means the real design question for a business application is: “is it acceptable for this data to be lost at the instant of a power failure?” A few seconds of log data might be tolerable. A confirmed order record probably is not. Reach for the tools in the next section only for the data you can’t afford to lose.

5. Building a Toolbox for “Definitely Written”

5.1. FlushFileBuffers — Force It Out Right Now

FlushFileBuffers writes all buffered data for the specified file through to the device. File-system metadata is always cached, so getting metadata itself onto disk reliably also requires a flush (or WRITE_THROUGH) — worth keeping in mind.¹² In .NET, FileStream.Flush(true) is the equivalent (Flush() alone only pushes .NET’s internal buffer out to the OS; the OS’s own cache is untouched).⁸

That said, the official documentation is explicit about one thing — calling it on every single write is inefficient. For many writes each needing durability, it says to use NO_BUFFERING combined with WRITE_THROUGH, covered below, instead.²

5.2. FILE_FLAG_WRITE_THROUGH — Removing Just the Delay

Open a file with FILE_FLAG_WRITE_THROUGH and every write is written to the cache as usual, but also written to disk immediately, without waiting for the lazy writer.¹ The point is that reads keep benefiting from the cache — a direct answer to “keep reads fast, but drop the write delay.”

5.3. FILE_FLAG_NO_BUFFERING — Bypassing the Cache

FILE_FLAG_NO_BUFFERING takes the system cache itself out of the picture for reads and writes. Every read and write becomes device I/O, on every call, with no cache in between.¹ But what it bypasses is only Windows’ system cache — as Figure 5 shows, the write cache inside the storage device is a separate stage. If you need resilience to power loss, you still need WRITE_THROUGH alongside it, or FlushFileBuffers. It’s a tool for bulk transfers of large amounts of data, or for database engines that manage their own buffering — but it comes with strict rules.³

• The size and file offset of every read and write must be an exact multiple of the volume’s sector size (for 512-byte sectors: 512, 1024, 1536, and so on).
• The buffer address must also be aligned to the physical sector size (bear in mind “Advanced Format” disks with 4096-byte physical sectors).
• Metadata continues to be cached regardless, so full durability still requires WRITE_THROUGH alongside it, or FlushFileBuffers.¹²

These “rules” are exactly where people trip up first when they think slapping on a flag is all it takes. Read or write without respecting alignment, and the call fails with ERROR_INVALID_PARAMETER (87). Here’s the summary of the three things you need to get right.³

The third one is the one people most often overlook. Addresses returned by malloc, new, or a C# array carry no guarantee of alignment to a sector boundary. Using VirtualAlloc, which allocates on page boundaries, also satisfies the requirements of “Advanced Format” disks with 4096-byte physical sectors along the way. Here’s a minimal shape:

// C++ / Win32. Error handling kept to a minimum.
DWORD sectorsPerCluster = 0, bytesPerSector = 0, freeClusters = 0, totalClusters = 0;
if (!GetDiskFreeSpaceW(L"C:\\", &sectorsPerCluster, &bytesPerSector,
                       &freeClusters, &totalClusters))
{
    return GetLastError();
}

// Make the read/write unit an exact multiple of the sector size (roughly 1MiB here)
const DWORD chunk = (1024 * 1024 / bytesPerSector) * bytesPerSector;

// Take a buffer aligned to a page boundary (malloc/new give no such guarantee)
BYTE* buffer = static_cast<BYTE*>(
    VirtualAlloc(nullptr, chunk, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE));
if (buffer == nullptr) { return GetLastError(); }

HANDLE h = CreateFileW(L"C:\\temp\\big.bin", GENERIC_READ, FILE_SHARE_READ, nullptr,
                       OPEN_EXISTING, FILE_FLAG_NO_BUFFERING, nullptr);
if (h == INVALID_HANDLE_VALUE)
{
    // GetLastError returns the result of the "most recent Win32 call". Calling
    // VirtualFree first would overwrite the real reason CreateFileW failed
    // (access denied, missing path, etc.) with the result of the cleanup call,
    // leaving only an uninformative error code.
    const DWORD err = GetLastError();
    VirtualFree(buffer, 0, MEM_RELEASE);
    return err;
}

// Advancing chunk bytes at a time keeps both size and offset aligned
DWORD read = 0;
while (ReadFile(h, buffer, chunk, &read, nullptr) && read > 0)
{
    // Process the first read bytes of buffer
    // (read < chunk at the end of the file — that's expected)
}

CloseHandle(h);
VirtualFree(buffer, 0, MEM_RELEASE);

Note also that .NET’s FileOptions has no value corresponding to FILE_FLAG_NO_BUFFERING. If you genuinely need it, you’ll be calling CreateFile directly, and in that case you still have to satisfy the alignment requirements above yourself. Approach it as “NO_BUFFERING because I manage my own buffering,” not “NO_BUFFERING because I want it to be fast.”

5.4. A Summary of When to Use What

flowchart TB
    A["App buffer"]
    B["System file cache<br/>dirty pages"]
    C["Cache inside the disk device"]
    D[("Non-volatile storage medium")]
    A -->|"Default WriteFile - success is returned once it gets here"| B
    B -->|"lazy writer, every second / WRITE_THROUGH, immediately"| C
    C -->|"Device timing /<br/>FlushFileBuffers demands a full write-through"| D
    A -.->|"NO_BUFFERING skips the cache and goes straight here"| C

Figure 5: The layers data passes through, and how far each tool pushes it. Don’t forget that final stage, “the cache inside the disk device”

The order to choose in is two steps: first decide how much can be lost at the instant of a power failure, then check how much slower you can afford for that. Don’t just skim the table top to bottom — walk this branch instead.

flowchart TB
    S["About to write this data"]
    Q1{"Is it acceptable to lose it<br/>at the instant of a power loss or blue screen"}
    A0["Stay with the default, cache enabled<br/>the fastest option, most I/O belongs here"]
    Q2{"Is what cannot be lost<br/>a milestone, or every single record"}
    A1["FlushFileBuffers at each milestone<br/>Flush(true) in .NET<br/>cost - only the wait at the milestone"]
    Q3{"Do you manage your own buffering<br/>and can you meet the alignment requirements from 5.3"}
    A2["FILE_FLAG_WRITE_THROUGH<br/>writes straight to disk on every write<br/>reads stay fast through the cache"]
    A3["FILE_FLAG_NO_BUFFERING<br/>plus FILE_FLAG_WRITE_THROUGH<br/>the form of frequent durability the official docs recommend"]
    S --> Q1
    Q1 -->|"acceptable<br/>e.g. the last few seconds of a log"| A0
    Q1 -->|"not acceptable"| Q2
    Q2 -->|"a milestone<br/>e.g. a transaction being finalised"| A1
    Q2 -->|"every single record"| Q3
    Q3 -->|"no, an ordinary app"| A2
    Q3 -->|"yes, a database engine etc."| A3

Figure 6: How to choose a tool. The first branch is your reliability requirement, the second is the cost you can bear. There’s no path for “FlushFileBuffers on every single record,” because, as we saw in 5.1, the official documentation calls that inefficient

Three practical patterns worth naming.

1. “Write to a temp file, flush, then rename” is the standard way to avoid leaving a half-written file behind. Write the full contents first, then commit by name — this atomic hand-off is covered in detail in “The Fundamentals of Exclusive Control for File Integration.”
2. Letting a database handle it is also a perfectly good design. For how SQLite builds durability out of WAL and flushing, see “Using SQLite in a Business Application from C#.” “Don’t write your own flush strategy” is always an option on the table.
3. When benchmarking, suspect the cache. A “reads are suspiciously fast” measurement is usually catching cache hits from the second pass onward. Good measurement practice is covered in “How to Correctly Compare Program Speed Across Versions on Windows.”

And don’t forget that final stage in Figure 5 — the cache inside the disk device itself. FlushFileBuffers demands a write-through all the way to there, but with USB drives and external disks, the device’s own write-cache policy (“Quick removal” versus “Better performance”) comes into play. See also “How to Work With USB Devices from a Windows App” for handling removable devices.

6. Coherency With Memory-Mapped Files

Back in Part 1, hearing that “the cache is, at bottom, a file mapping” probably left some readers wondering: so does a view I map myself with MapViewOfFile fight with the cache used by ReadFile/WriteFile?

It doesn’t. Because they sit on top of the same mechanism. A file mapping object is backed by a file, and evicting a page is carried out as a write-back to that file. Even when multiple processes create views of the same local file, what they see is coherent.⁴

flowchart TB
    subgraph P1["Process A's address space"]
        V1["MapViewOfFile view"]
    end
    subgraph SYS["System address space"]
        SC["Cache manager's view<br/>the slot used by ReadFile/WriteFile"]
    end
    PAGES["The same physical pages<br/>memory backed by the file"]
    DISK[("File on disk")]
    V1 --> PAGES
    SC --> PAGES
    PAGES --> DISK
    NB["I/O on a FILE_FLAG_NO_BUFFERING handle<br/>sits outside this sharing, straight to disk"]
    NB -.-> DISK

Figure 7: Both the mapped view and the cache are looking at the same “file-backed pages.” The only thing outside that circle is NO_BUFFERING

Two things to watch for.

• I/O through FILE_FLAG_NO_BUFFERING sits outside this coherency. Reads and writes that bypass the cache are not reconciled against content seen through a mapped view or the cache. Mix the two, and you have to maintain consistency yourself.
• Persisting a mapped view is a two-step process. FlushViewOfFile starts writing out the dirty pages in the given range, but it does not write metadata, and it does not wait for the physical write from the disk device’s own cache. To reliably get it onto disk, call FlushFileBuffers after FlushViewOfFile.⁵

The practical side of file mapping as shared memory — named sharing, synchronisation, common failure patterns — is covered in “Pitfalls and Best Practices for Shared Memory.”

7. Fast I/O — Collecting on Part 1’s Homework

In two lines first: fast I/O is a shortcut built for synchronous reads and writes against a file that’s already in the cache; it exchanges data directly with the cache without building an IRP (an I/O request packet — the container the kernel hands a request to a driver in). The reason Procmon’s Operation column shows a mix of IRP_MJ_READ and FASTIO_READ is that the same “read” can go through either the normal path or this shortcut.

Part 1, Section 5.2 said “not every I/O becomes an IRP.” Here’s the answer.

For a file sitting in the cache, reads and writes are known to be satisfiable as a plain memory copy against the cache, without going to the trouble of building an IRP and sending it down the device stack. So Windows provides a shortcut called fast I/O for synchronous I/O against a cached file: it builds no IRP, calls the file system’s “fast I/O entry points” directly, and copies straight from the cache manager.⁶ When fast I/O can’t handle the request (not in the cache, a lock is involved, a filter intervenes, and so on), it falls back to the normal IRP path. Note that this is a fast path for synchronous requests specifically — a cache hit does not always mean fast I/O. Operations on an asynchronous (FILE_FLAG_OVERLAPPED) handle can still go through the IRP path even when they complete on the spot from the cache (Part 2, Section 5).

flowchart TB
    REQ["Synchronous reads/writes on a cache-enabled handle"]
    Q{"Can fast I/O handle it<br/>e.g. it is already in the cache"}
    FAST["Fast I/O<br/>copies directly with the cache, builds no IRP<br/>shows as FASTIO_ in Procmon"]
    IRP["The normal path<br/>builds an IRP and sends it down the device stack<br/>the world of Figure 6 in Part 1"]
    REQ --> Q
    Q -->|can| FAST
    Q -->|cannot| IRP

Figure 8: The fast-I/O branch. This is why FASTIO_READ and IRP_MJ_READ show up mixed together in Procmon

That explains why rows starting with FASTIO_ turned up mixed in among the Procmon observations in Part 1, Section 7. For a cache-hit read, even an IRP is a luxury. This path’s existence also matters for the filter drivers covered in Part 6 (minifilters can hook into fast I/O too).

8. Summary

• Windows’ file cache is write-back, and at bottom is a mapping of 256KB sections of a file. Cache-enabled reads and writes are memory copies against a slot.¹
• Reads are speculated on by read-ahead, and SequentialScan/RandomAccess are hints for it.¹
• Writes are caught up by the lazy writer, once per second. Data survives an application death; only the dirty portion is lost if the OS itself dies. The design question is “can this data be lost at the instant of a power failure?”¹
• The tools for writing reliably are FlushFileBuffers (committing at a milestone) / WRITE_THROUGH (every write) / NO_BUFFERING (bypasses the cache, with alignment requirements). Flushing every time is inefficient; for frequent durability, the official recommendation is combining NO_BUFFERING with WRITE_THROUGH. Remember that metadata is always cached.²³¹
• A mapped view and the cache share the same pages and stay coherent. The one thing outside that is NO_BUFFERING. Persisting a mapping is a two-step process: FlushViewOfFile followed by FlushFileBuffers.⁴⁵
• Synchronous reads and writes that hit the cache skip even the IRP, via fast I/O — the real identity of the FASTIO_ entries seen in Part 1’s Procmon trace.⁶

Next up is Part 5, “NTFS Internals — Understanding the File System Through the MFT.” Up to now we’ve treated a file as “an offset and a run of bytes”; from here we go down beneath that, into how NTFS actually lays data out on disk — the MFT, multiple data streams, the journal, hard links — the static structure sitting on the disk itself.

Related Articles

• The Depths of Windows I/O (Part 1) — Every Read and Write Becomes an IRP: The Big Picture of the I/O System
• The Depths of Windows I/O (Part 2) — Synchronous and Asynchronous I/O: What OVERLAPPED Really Means
• The Depths of Windows I/O (Part 3) — I/O Completion Ports (IOCP) and the .NET Thread Pool: The Basement Under async/await
• Mutual Exclusion Fundamentals for File-Based Integration - Best Practices for File Locks and Atomic Claims
• Shared Memory Pitfalls and Practical Best Practices
• Using SQLite from C# in Business Apps — WAL Mode, Exclusive Locking, Corruption Countermeasures, and When to Reach for EF Core
• How to Correctly Compare the Speed of Different Program Versions on Windows
• How to Work with USB Devices from a Windows App — Choosing Between virtual COM, HID, WinUSB, and Vendor SDKs

Related Consulting Areas

KomuraSoft LLC handles design and troubleshooting of file I/O in Windows business applications — cases like “data I thought was saved has disappeared” or “file writes are slow, or suspiciously fast.”

• Windows Application Development
• Bug Investigation and Root Cause Analysis
• Legacy Asset Utilisation and Migration Support
• Contact Us

References

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Frequently Asked Questions

Common questions about the topic of this article.

Once WriteFile returns success, has the data been written to disk?

By default, no. Windows' file cache is a write-back cache, and WriteFile returns success as soon as it has copied the data into the system file cache. The actual write to disk is done afterwards by the cache manager's lazy writer, which runs once per second. What matters is the difference between kinds of failure. If the application's process crashes, data that made it into the cache is not lost, because the OS goes on to write it back later as long as the OS itself is alive. If the whole OS goes down instead — a power loss, a blue screen — dirty cache data that had not yet been written back is lost. The accurate way to read "WriteFile succeeded" is not "it has been made durable" but "it has been handed off to the OS".

How do I make sure data actually reaches the disk?

There are three tools. First, FlushFileBuffers, which writes all buffered data and metadata for that file through to the device (in .NET, FileStream.Flush(true) is the equivalent). Second, FILE_FLAG_WRITE_THROUGH, which writes to the cache on every write while also writing to disk immediately. Third, FILE_FLAG_NO_BUFFERING, which bypasses the cache altogether. Microsoft's own documentation notes that calling FlushFileBuffers on every write is inefficient, and that applications needing reliable durability with frequent writes should combine FILE_FLAG_NO_BUFFERING with FILE_FLAG_WRITE_THROUGH instead. Every one of these gives up some of the cache's benefit in exchange for speed, so the practical rule of thumb is not to slap them on everything but to reserve them for writes you genuinely cannot afford to lose.

What's the difference between FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING?

WRITE_THROUGH means "still write to the cache, but also write to disk before completion." Reads keep benefiting from the cache; only the lazy-writer delay on writes is removed. NO_BUFFERING means "reads and writes bypass the system cache entirely," so both become device I/O on every call (though what it bypasses is only Windows' own cache — it doesn't skip a write cache sitting inside the storage device itself). In exchange, it comes with strict constraints: the size and file offset of every read or write must be an exact multiple of the volume's sector size, and the buffer address itself must be aligned to a physical sector boundary. Also, even with NO_BUFFERING, file-system metadata continues to be cached, so making metadata durable still requires FlushFileBuffers or WRITE_THROUGH on top. It's typically used by software that manages its own buffering, such as database engines; ordinary applications should generally reach for WRITE_THROUGH or FlushFileBuffers first.

Task Manager shows little free memory — is that the file cache's fault?

Often, yes, and that's normal behaviour. Windows actively uses free physical memory as file cache, so copying a large file or doing a lot of reading and writing will swell the cache and make memory usage look higher. That said, most of the pages the cache is using are of a kind that can be reclaimed fairly promptly the moment an application asks for memory, which needs to be distinguished from genuinely running out of memory. When you suspect a real shortage, it's more useful to look at committed memory and hard-fault frequency than at the apparent amount of free memory alone.

If I touch the same file through a memory-mapped view and through ReadFile/WriteFile, can the contents get out of sync?

Not against ordinary cache-enabled I/O. Windows' own cache is itself implemented as a file mapping, so a mapped view and the cache for the same local file share the same data — a change made through one is visible through the other. Multiple processes that create views from the same file mapping object also see coherent data. However, reads and writes on a handle opened with FILE_FLAG_NO_BUFFERING bypass the cache and fall outside this coherency. Also, to reliably persist changes made through a mapped view, FlushViewOfFile alone is not enough — it does not write metadata and does not wait for a hardware cache — so you need to call FlushFileBuffers after FlushViewOfFile.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.