KomuraSoft LLC
Contact
Menu

A Practical Guide to FileSystemWatcher - Handling Missed and Duplicate Events

· Updated: · · FileSystemWatcher, C#, .NET, Windows Development, File Integration, Design

Revision history (1 updates, last updated Sep 1, 2026)

A log of the changes made to this article. Where a pre-update version was archived, it stays readable at a permanent DOI link.

Retranslated as a full translation of the Japanese original. The previous English version was an abridgement that carried only part of the source, so sections, tables, Mermaid diagrams, figure captions and FAQ entries were missing. All of them have been restored to match the Japanese original, and the technical claims are the same as in the Japanese version. Read the version before this update (DOI: 10.5281/zenodo.21614465)
First published
Cite this article(DOI: 10.5281/zenodo.21614464)

This article is archived on Zenodo. Below are both the DOI that always resolves to the latest version and the DOI pinned to the version you are reading.

Go Komura (2026). A Practical Guide to FileSystemWatcher - Handling Missed and Duplicate Events. KomuraSoft LLC. https://doi.org/10.5281/zenodo.21614464 https://comcomponent.com/en/blog/2026/03/10/000-filesystemwatcher-safe-basics/

DOI (latest version)
10.5281/zenodo.21614464
DOI (this version)
10.5281/zenodo.22217124

FileSystemWatcher is the first API that comes up when monitoring file changes in .NET on Windows. It conveniently delivers file and directory creations, changes, deletions, and renames as events - but if you treat Created or Changed as completion notifications, you will quite routinely get burned by missed events, duplicate notifications, and reading half-written files.

In this article, we organize how to use FileSystemWatcher and its pitfalls, assuming mainly file-based integration with .NET on Windows. For the underlying mutual-exclusion concepts, see also Mutual Exclusion Fundamentals for File-Based Integration - Best Practices for File Locks and Atomic Claims.

In reality, Created can fire while a file is still being copied, and Changed is by no means guaranteed to fire only once. When changes concentrate in a short window, the internal buffer can overflow and individual changes get dropped.

So the core of the design is this.

  • Notifications are triggers
  • The truth lives in a directory rescan
  • Ownership comes from an atomic claim
  • Idempotency catches whatever is left

In the body of the article, we walk through the traps of wiring FileSystemWatcher into file integration with this mindset.

The code in this article is published on GitHub as a complete buildable and runnable sample set (a library, a console demo that runs against a temporary directory, and unit tests that actually create and modify files to verify the events).

filesystemwatcher-safe-basics - komurasoft-blog-samples (GitHub)

Target Readers and Prerequisites

This article is written for developers who write code on .NET for Windows that watches an incoming directory and ingests files. The code samples assume C# / .NET 8 or later, but the reasoning itself is language-independent.

It reuses the same vocabulary as the previous article linked above (mutual exclusion in file-based integration). Terms like claim, idempotency, manifest, and bundle appear from chapter 4 onward without further explanation, so here is a one-line summary of each so you can follow along even without having read the previous article.

Terms to Know Up Front

Term Meaning
claim Taking ownership of a file - “I am the one processing this” - in a way no other worker can cut into. The implementation is a rename from incoming/ to processing/<worker>/, and only the single process whose rename succeeds becomes the owner (4.3)
idempotency The property that processing the same target two or more times produces the same result as processing it once. Since duplicate notifications and rescans are a given, this is where everything is ultimately absorbed (4.5)
manifest A small file placed alongside the payload that describes its contents. Recording the record count, a hash, an IdempotencyKey, and similar lets the receiver decide whether this has already been processed
bundle One unit of transfer packaged together. Putting the payload, the manifest, and any auxiliary files in a single directory lets you claim that whole directory with one rename (4.3)
full rescan Ignoring events entirely and re-enumerating the watched directory from scratch to identify what may be processed (4.4)
overflow The internal buffer of FileSystemWatcher filling up and losing individual notifications. It is reported through the Error event (2.3)
ready The state in which a file can be judged safe to read. Determined by the presence of a final name or a done / manifest file, not by guesswork (4.2)

Table of Contents

  1. The Conclusion First (In One Line)
    • 1.1. A Minimal Working Example
  2. Common Misconception Patterns With FileSystemWatcher (Diagrams)
    • 2.1. Treating Created as a Completion Notification
    • 2.2. Trusting the Count and Order of Changed
    • 2.3. Losing Changes to Internal Buffer Overflow
  3. Anti-Patterns
    • 3.1. Processing Directly Inside the Event Handler
    • 3.2. Trying to Reconstruct the True State From the Event Stream
    • 3.3. Treating “No More Changed” as Completion
    • 3.4. Believing a Bigger InternalBufferSize Solves It
    • 3.5. Logging Error and Ignoring It
  4. Best Practices
    • 4.1. Fold Notifications Into “Rescan Requests”
    • 4.2. Make Completion Explicit on the Sender Side
    • 4.3. The Receiver Takes a Claim Atomically
    • 4.4. Full Rescan on Startup / Overflow / Reconnection
    • 4.5. Assume Idempotency
  5. Pseudocode (Excerpts)
    • 5.1. The Typical Failure Pattern
    • 5.2. An Example in the Right Direction (Roughly Sketched)
  6. A Rough Guide to Choosing
  7. Conclusion
  8. References

Knowledge map for this article

This article proposes a design that avoids treating the Created and Changed events of FileSystemWatcher as completion notifications, avoids losing notifications to internal buffer overflow, and avoids any design that tries to reconstruct state from the event stream, and instead collapses every notification into a single kind of rescan request and confirms the actual files with a full rescan. The sender states completion explicitly with temp-to-rename plus a done file or a manifest, and the receiver takes an atomic claim on each ready candidate found by the rescan and absorbs repeated visits with idempotency. For requirements where the process cannot stay running at all times, or where missing an event is not acceptable, it positions the USN change journal as a further option.

Practical guide to FileSystemWatcherDiagram showing that a FileSystemWatcher notification is only a hint of change rather than a completion signal, that notifications should be collapsed into rescan requests combined with a full rescan and a claim, and how notifications missed through internal buffer overflow relate to the USN change journal as an alternativemay causeusesrecommended forrecommended fornot recommended formay causemay causemay causerecommended forusesusesrecommended forrecommended forrecommended formay causepreventsusesusesrecommended forshould come beforeshould come beforemay causemitigatesrecommended forrequiresrequiresFileSystemWatcherFull Rescan (Directory Re-enumeration)Event Loss from Buffer OverflowPeriodic Directory EnumerationMissed Change NotificationsInternalBufferSize TuningIgnored Error Event Anti-PatternMistaking Created for completionReading a Partially Written FileSender-Side Completion SignalingTemp-Then-Rename Publish Patterndone/manifest FileCoalescing Notifications into Rescan RequestsTrusting Changed Event Count and OrderState Reconstruction from Events Anti-PatternDuplicate Processing and Lost UpdatesAtomic ClaimBundle (Exchange Unit Directory)Idempotent Processing DesignMissed Changes During Watcher DowntimeUSN Change JournalNTFSAdministrator Privileges

In the diagram a solid line marks a relation that always holds and a dashed line marks a conditional one (the conditions are given per relation on the detail page). The full list of relations (26 in total, with evidence and certainty) and the definitions of the main concepts are collected on the knowledge map detail page (in Japanese). Data: JSON-LD / Turtle

1. The Conclusion First (In One Line)

  • FileSystemWatcher events are not completion notifications - they are hints that something changed
  • Created / Changed / Renamed can duplicate, arrive in an order you did not expect, and get dropped on overflow
  • Event handlers are more stable when they do no heavy work and only enqueue a rescan request
  • Completion detection should be made explicit via temp -> close -> rename / replace or done files / manifests
  • With multiple workers, you must take a claim atomically before reading
  • Tuning InternalBufferSize is an aid. In the end, full rescans and idempotency are what work

In short: do not treat FileSystemWatcher as “a truthful history stream.” Things break far less if notifications remain nothing more than a signal of “time to go look.”

The core of the design in this articleShows the core of the design in this article - keep notifications as nothing more than a trigger, confirm the truth with a directory rescan, take ownership through an atomic claim, and absorb duplicates with idempotency at the end.Notifications are triggersThe truth lives in a directory rescanOwnership comes from an atomic claimIdempotency catches whatever is left

Figure 1: The core of the design. Do not treat events as a truthful history; keep them as a signal that says time to go look.

1.1. A Minimal Working Example

For anyone who has never touched FileSystemWatcher, here is the smallest form that covers the happy path only. Everything from the next chapter onward is about the traps that start right where these ten lines “happen to work.”

// C# / .NET 8 console app. The minimal form that only confirms notifications arrive
using System.IO;

using var watcher = new FileSystemWatcher(@"C:\incoming")
{
    Filter = "*.csv",
    NotifyFilter = NotifyFilters.FileName | NotifyFilters.LastWrite,
};

watcher.Created += (_, e) => Console.WriteLine($"Created: {e.FullPath}");
watcher.Changed += (_, e) => Console.WriteLine($"Changed: {e.FullPath}");
watcher.Renamed += (_, e) => Console.WriteLine($"Renamed: {e.OldFullPath} -> {e.FullPath}");
watcher.Error += (_, e) => Console.WriteLine($"Error: {e.GetException().Message}");

watcher.EnableRaisingEvents = true; // Watching starts here
Console.WriteLine("Press Enter to exit");
Console.ReadLine();

Even in this minimal form, three things are worth fixing in your head from the start.

  • Not a single event arrives until EnableRaisingEvents = true. Registering handlers alone does nothing
  • The lifetime of the watcher is the lifetime of the app. Once a local variable goes out of scope and the object is disposed, notifications stop there. For a long-running process, hold it somewhere that stays alive, such as a field
  • The default NotifyFilter is the combination LastWrite | FileName | DirectoryName (see FileSystemWatcher.NotifyFilter Property in chapter 8, References). Stating explicitly what you want to pick up saves confusion when you reread the code later

And the important part: this code only confirms that events arrive. It tells you nothing about whether the file may be read at the time of Created, or whether notifications were dropped. That is where the real subject begins.

2. Common Misconception Patterns With FileSystemWatcher (Diagrams)

2.1. Treating Created as a Completion Notification

This is the most obvious landmine. During copies and transfers, Created fires the moment the file is created, and one or more Changed events may follow afterwards.

ReceiverFileSystemWatcherwatched dirSenderReceiverFileSystemWatcherwatched dirSenderCopy still in progressMissing rows / corrupt JSON / corrupt ZIPCreates orders.csvCreatedOnCreatedOpens and reads orders.csvWrites the restChangedChanged

Figure 2: Created fires even mid-copy. Reading as soon as it arrives means grabbing broken data.

Created may mean “the name became visible,” but it does not guarantee “it is now safe to read.” If you equate the two, you step on the same mine as section 2.1 of the previous article, just by a different route.

2.2. Trusting the Count and Order of Changed

Changed is not guaranteed to fire exactly once. Even ordinary operations like moving or saving can show up as multiple events. On top of that, you can also pick up the touches of antivirus software and indexers.

FileSystemWatcherAV / indexerwatched dirSaving appFileSystemWatcherAV / indexerwatched dirSaving appNot guaranteed to be once-only or in this orderStarts saving report.xlsxCreatedChangedRename from a temp fileRenamedChangedScan / attribute accessChanged

Figure 3: Even an ordinary save splits into several events, mixed with whatever external processes touch. Neither the count nor the order can be relied on.

Expectations like “one Changed means done” or “nothing touches the file after Renamed” are quite precarious.

Additional notes:

  • A file rename can produce a Changed event
  • RenamedEventArgs.Name can be null when the OS cannot correlate the old/new names
  • Hidden files are not exempt. “It’s a hidden temp name, so it won’t be seen” does not hold
  • If the watched directory itself is renamed, that change is not reported

2.3. Losing Changes to Internal Buffer Overflow

FileSystemWatcher has an internal buffer. When changes concentrate in a short window, it overflows and individual notifications get dropped.

YesNoMany changes in a short windowNotifications pile up in the internal bufferCan processing keep up?Process individual events in orderOverflowError eventStop trusting the completeness of the per-event historyFull rescan of the directory

Figure 4: When a notification burst exceeds the internal buffer it overflows, and the completeness of the individual event stream falls apart.

The important point here is that “an overflow means losing just one event” is not guaranteed. The completeness of the entire individual-event stream becomes suspect, so it is best to simply re-examine the whole picture.

3. Anti-Patterns

3.1. Processing Directly Inside the Event Handler

This puts too much weight - completion detection and ownership acquisition - on the events themselves.

watcher.Created += (_, e) =>
{
    using var stream = File.OpenRead(e.FullPath);
    Import(stream); // May still be mid-copy
};

watcher.Error += (_, e) =>
{
    Console.WriteLine(e.GetException()); // Just printing it
};

There are two problems.

  • At the time of Created, the contents may be incomplete
  • There is no recovery from failures or overflow

An event handler is at its best when it just raises a rescan request and returns immediately. If you start heavy I/O or DB updates here, you only make things worse for yourself during bursts.

What separates a light event handler from a heavy oneShows that an event handler should raise a rescan request and return immediately, and that starting heavy I/O or DB updates inside the handler carries both the risk of reading incomplete content and the problem of falling behind during bursts.Event handlerRaise a rescan request and return immediatelyHeavy I/O or DB updates inside the handlerRisk of reading incomplete contentCannot keep up during bursts

Figure 5: Keep the handler light. Do not make events carry completion detection and ownership acquisition.

3.2. Trying to Reconstruct the True State From the Event Stream

The design of “add to a dictionary on Created, update on Changed, remove on Deleted, re-key on Renamed” looks clean at first glance. But once duplicates, splits, overflow, and external interference enter, the bookkeeping gradually stops adding up.

switch (e.ChangeType)
{
    case WatcherChangeTypes.Created:
        state[e.FullPath] = Pending;
        break;
    case WatcherChangeTypes.Changed:
        state[e.FullPath] = Modified;
        break;
    case WatcherChangeTypes.Deleted:
        state.Remove(e.FullPath);
        break;
}

Rather than struggling in this direction, it is stronger to re-check the actual files on disk each time. What matters in file integration is correctly finding what may be processed right now - not faithfully reconstructing the event history.

Reconstructing from events versus checking the actual filesShows that a design reconstructing state from the event stream stops adding up under duplicates, splits, overflow, and external interference, so re-checking the actual files on disk each time and correctly finding what may be processed right now is stronger.Reconstruct state from the event streamBookkeeping breaks under duplicates, splits, and overflowCheck the actual files on disk each timeCorrectly find what may be processed

Figure 6: The goal is not reproducing the event history but finding what may be processed right now.

3.3. Treating “No More Changed” as Completion

This design smells just like the previous article’s “size stopped changing, so it’s done.” It looks convenient, but it determines completion by guessing.

if (lastChangedAt + TimeSpan.FromSeconds(10) < DateTime.UtcNow)
{
    return Ready;
}

Cases where this fails include:

  • A large file copy pauses midway
  • The sending app saves in multiple stages
  • Notifications appear delayed over a network share
  • An external process rewrites attributes or timestamps afterwards

Completion is more stable when stated explicitly rather than guessed.

The danger of inferring completion from silenceShows that guessing completion once Changed has been quiet for a while misjudges paused copies, multi-stage saves, delayed notifications, and later attribute rewrites, so having the sender declare completion explicitly is more stable.Guess completion once Changed stopsMisjudged when a copy pausesMisjudged on multi-stage saves and delayed notificationsThe sender declares completion explicitlyStable without relying on guesswork

Figure 7: Silence is no evidence of completion. Decide completion by explicit declaration, not by inference.

3.4. Believing a Bigger InternalBufferSize Solves It

Tuning InternalBufferSize matters, but it is not the heart of the design.

  • The default is 8192 bytes
  • It cannot go below 4096 bytes, and cannot exceed 64 KB
  • The buffer uses non-paged memory, so increasing it is not as casual as it sounds

In other words, even at 64 KB, a notification burst beyond it ends the story. And it does nothing at all for the “is this a completion notification?” problem.

Before enlarging the buffer, there are things to do first.

  • Narrow the watch scope with Filter / Filters
  • Keep NotifyFilter to the necessary minimum
  • Do not set IncludeSubdirectories to true carelessly
  • Keep the event handlers lightweight
  • Add full rescans and idempotency
What to do before enlarging the bufferShows the order to work in - since raising InternalBufferSize to 64 KB still drops events when a burst exceeds it, first narrow the watch scope with Filter and NotifyFilter, keep the handlers light, and add full rescans and idempotency.Things to do firstNarrow the scope with Filter and NotifyFilterKeep the handlers lightFull rescans and idempotencyTuning InternalBufferSizeKeep it as a last-resort aid

Figure 8: Enlarging the buffer is not the heart of the design. Narrowing the scope and building recovery come first.

3.5. Logging Error and Ignoring It

Error is not the kind of notification you can “see occasionally and shrug off.” Buffer overflows and failures to continue watching surface here.

watcher.Error += (_, e) =>
{
    _logger.LogError(e.GetException(), "watcher error");
    // Ending here means noticing the loss but never recovering
};

At a minimum, you want to go this far.

  • Request a full rescan
  • If continued watching is in doubt, consider recreating the watcher
  • Make reprocessing idempotent, on the assumption that events were lost

4. Best Practices

4.1. Fold Notifications Into “Rescan Requests”

Wiring Created / Changed / Deleted / Renamed / Error each directly into separate business logic ruins clarity. First fold them all into one kind of signal: “go look.”

Created / Changed / Deleted / Renamedscan requestError / overflowstartupRescan the directoryEnumerate ready candidatesAttempt a claim

Figure 9: Every notification, and startup too, folds into a single kind of scan request; the rescan then looks for ready candidates and attempts a claim.

Implementation points:

  • In the event handler, do little more than set dirty = true and raise a signal
  • Concentrate scanning in a single worker
  • During bursts, coalesce for around 100-300 ms, then scan once
  • If more notifications arrive during a scan, scan once more afterwards

The 100-300 ms in that third bullet is not a number backed by a standard or by official documentation - it is a starting value from the author’s operational experience. In practice you are better off measuring these two things before deciding.

What to look at How to decide
The time one scan takes If the wait is shorter than this, the next scan request simply piles up before the current scan finishes. Use something at or above the scan time as the lower bound
The detection delay you can tolerate The wait translates directly into detection delay. If the requirement is “processed within n seconds of being dropped in,” cap the wait at a fraction of that

For example, if one scan finishes in 50 ms and detection within one second is acceptable, 100-300 ms fits comfortably. Conversely, if there are so many files that a single scan takes several seconds, reworking how the scan is built (narrowing the target set, looking only at done files, splitting subdirectories) is more effective than stretching the wait.

Done this way, whether five events arrive or fifty, the final action is unified: “look at the actual files and find what is ready.”

4.2. Make Completion Explicit on the Sender Side

If you also control the sender, fixing the publishing protocol beats straining over completion detection on the FileSystemWatcher side.

The proven route, once again, is:

  • Write the full content under a temp name
  • close it
  • rename / replace on the same file system
  • If needed, place a done file / manifest last
Write the full content to data.tmpflush / closerename / replace to data.csvPlace data.done / manifest.jsonReceiver watches only final names or done files

Figure 10: The sender writes the full content to a temp file, closes it, publishes by rename, and if needed places done / manifest last.

Same as the previous article, but this is where the real payoff is. The right way to see FileSystemWatcher is not as a tool that invents completion, but as a tool that notices explicitly declared completion sooner.

4.3. The Receiver Takes a Claim Atomically

Even when a rescan finds a ready candidate, going straight in to read lets multiple workers grab it simultaneously. So take a claim atomically before processing.

processing/worker2processing/worker1incomingscannerprocessing/worker2processing/worker1incomingscannerOnly the one that succeeds first holds ownershipFinds order-123rename order-123rename order-123

Figure 11: Even when several workers find the same candidate, only the one whose rename succeeds holds ownership.

As mentioned in the previous article, the incoming -> processing/<worker>/ rename is the clearest. Bundling the payload + manifest + auxiliary files into one directory is especially convenient, since you can then claim per bundle.

incoming/
  order-123/
    payload.csv
    manifest.json

With this, a single rename of the bundle directory takes ownership.

4.4. Full Rescan on Startup / Overflow / Reconnection

This is quite important.

  • Files placed before the app started are not picked up by events
  • Once an overflow occurs, the individual event stream becomes hard to trust
  • With network shares and transient disconnections in play, it is safer to assume “something in between” was missed

So at least at these moments, a full rescan should be performed.

  • At startup
  • On receiving Error
  • Right after recreating the watcher
  • Periodically, as insurance, at a fixed interval

The philosophy here: “the watcher is a hint about deltas; the rescan is the recovery of consistency.”

When to run a full rescanShows that running a full rescan at four moments - at startup, on receiving Error, right after recreating the watcher, and periodically as insurance - recovers the changes that events cannot pick up.At startupfull rescanOn receiving ErrorRight after recreating the watcherPeriodically as insuranceRecovery of consistency

Figure 12: The watcher is a hint about deltas; the full rescan is the recovery of consistency. Always include these four moments.

4.5. Assume Idempotency

With FileSystemWatcher, you will end up examining the same target multiple times. That is not a bug - it is more stable to accept it as part of the design.

Concretely, it goes like this.

  • Put an IdempotencyKey in the manifest
  • If already processed, do not re-execute the side effects
  • Make archived / DB-recorded / sent statuses verifiable
  • Ensure that a full rescan amounts to nothing more than “safely looking at the same things again”

Trying to build exactly-once out of events alone gets painful fast. Accepting at-least-once and closing the loop with idempotency is the stronger position in practice.

How to absorb duplicates by designShows that examining the same target multiple times should be accepted as part of the design rather than treated as a bug, and that checking an IdempotencyKey in the manifest against what has already been processed keeps side effects from running twice, making rescans safe.Examine the same target multiple timesAccept it as part of the designCheck processed state with an IdempotencyKeyDo not re-execute the side effectsA full rescan is just a safe second look

Figure 13: Do not build exactly-once out of events; accept at-least-once and close the loop with idempotency.

5. Pseudocode (Excerpts)

5.1. The Typical Failure Pattern

using var watcher = new FileSystemWatcher(incomingDir)
{
    Filter = "*.csv",
    IncludeSubdirectories = false,
    EnableRaisingEvents = true,
    InternalBufferSize = 64 * 1024
};

watcher.Created += (_, e) =>
{
    // Assumes Created = completion notification
    ProcessFile(e.FullPath);
};

watcher.Changed += (_, e) =>
{
    // It keeps firing, so just process again
    ProcessFile(e.FullPath);
};

watcher.Error += (_, e) =>
{
    Console.WriteLine(e.GetException());
    // No recovery
};

There are four problems.

  • Created / Changed are wired directly into business processing
  • There is no completion detection
  • No full rescan on overflow
  • No mechanism to stop processing the same file repeatedly

5.2. An Example in the Right Direction (Roughly Sketched)

private readonly SemaphoreSlim _scanSignal = new(0, int.MaxValue);
private int _scanRequested = 0;
private int _fullRescanRequested = 0;

void OnAnyChange(object? sender, FileSystemEventArgs e)
{
    RequestScan(full: false);
}

void OnRenamed(object? sender, RenamedEventArgs e)
{
    RequestScan(full: false);
}

void OnError(object? sender, ErrorEventArgs e)
{
    Log(e.GetException());
    RequestScan(full: true);
}

void RequestScan(bool full)
{
    if (full)
    {
        Interlocked.Exchange(ref _fullRescanRequested, 1);
    }

    if (Interlocked.Exchange(ref _scanRequested, 1) == 0)
    {
        _scanSignal.Release();
    }
}

async Task ScannerLoopAsync(CancellationToken cancellationToken)
{
    RequestScan(full: true); // startup scan

    while (!cancellationToken.IsCancellationRequested)
    {
        await _scanSignal.WaitAsync(cancellationToken);

        // Coalesce notification bursts a little
        await Task.Delay(TimeSpan.FromMilliseconds(200), cancellationToken);

        Interlocked.Exchange(ref _scanRequested, 0);
        bool full = Interlocked.Exchange(ref _fullRescanRequested, 0) == 1;

        foreach (var bundle in EnumerateReadyBundles(incomingDir, full))
        {
            var claimedPath = Path.Combine(processingDir, bundle.Name);

            if (!TryClaimByRename(bundle.Path, claimedPath))
            {
                continue; // Another worker claimed it first
            }

            var manifest = ReadManifest(Path.Combine(claimedPath, "manifest.json"));

            if (AlreadyProcessed(manifest.IdempotencyKey))
            {
                MoveToArchive(claimedPath, archiveDir);
                continue;
            }

            ProcessBundle(claimedPath);
            RecordProcessed(manifest.IdempotencyKey);
            MoveToArchive(claimedPath, archiveDir);
        }

        if (Volatile.Read(ref _scanRequested) == 1)
        {
            _scanSignal.Release(); // Do not drop notifications that arrived mid-scan
        }
    }
}

What matters in this example is the flow, not the fine API details.

  • Fold notifications into scan requests
  • Find what is ready by scanning
  • Take a claim
  • Check idempotency
  • Process, record, and move to the archive
The flow of the approach that worksShows the sequence the pseudocode represents - fold notifications into scan requests, find ready candidates by scanning, take a claim, check idempotency, then process, record, and move the result to the archive.Fold notifications into scan requestsFind what is ready by scanningTake a claimCheck idempotencyProcess, record, and move to the archive

Figure 14: This flow, not the fine API details, is the substance. Events are only a trigger.

The FileSystemWatcher events are nothing more than a trigger here.

Note that EnumerateReadyBundles / TryClaimByRename / ReadManifest / AlreadyProcessed and the like are functions named in this article to show the flow - they are not standard .NET APIs. A form that actually builds and runs (a library, a console demo that runs against a temporary directory, and unit tests that verify the events) is in the sample set linked at the top.

filesystemwatcher-safe-basics - komurasoft-blog-samples (GitHub)

6. A Rough Guide to Choosing

  • Single receiving worker / you can also fix the sender Start with temp -> close -> rename and a startup scan. That alone gets you quite far.

  • Multiple receiving workers Add the incoming -> processing claim rename on top of the above.

  • High-frequency, notification-heavy Narrow Filter / NotifyFilter / IncludeSubdirectories and minimize the event handlers. Tuning InternalBufferSize comes after that.

  • Overflows hurt / missed events are unacceptable Build on full rescans, and if that is still not enough, do not bet on FileSystemWatcher alone. If you are Windows-only, the USN change journal is also an option.

  • You cannot control how the other system writes Rather than papering over completion conditions with guesses, first consider whether the publishing protocol can be negotiated. If not, lower the guarantee level and lean into an idempotent receiving design.

The last two items are fairly important calls about when to walk away. FileSystemWatcher is useful, but it is not an all-powerful truth detector.

What is different about the USN change journal

The USN change journal is a record of changes that NTFS keeps per volume. Directory notifications like FileSystemWatcher can only be received if the application is running at the moment the change happens, but the change journal leaves the record on the volume side, so changes that occurred while the application was down can be read back later from the position (USN) you last read. Microsoft’s own documentation lists “the application has to be kept running at all times” as a weakness of directory notifications, and presents the change journal as the way around it.

The cost, on the other hand, goes up.

  FileSystemWatcher USN change journal
Unit of watching The specified directory (+ subdirectories) The whole volume. Narrowing to the range you need is on you
While the application was down Unknown. Fill the gap with a full rescan Can be read back from the record
Missed events Caused by internal buffer overflow Once the journal’s size limit is exceeded, the oldest records are dropped
What you need Only the .NET API A volume handle and FSCTL_* calls. Administrative operations such as creating or deleting the journal require administrator rights

In short, it is the option for when “cannot be kept running at all times” or “changes during downtime must also be picked up” enter the requirements. If neither applies, FileSystemWatcher plus a full rescan is the more straightforward implementation.

The difference between FileSystemWatcher and the USN change journalShows the difference that FileSystemWatcher cannot know about changes made while the application was down and fills the gap with a full rescan, whereas the USN change journal keeps the record on the volume side so changes during downtime can be read back from the position last read.FileSystemWatcherChanges during downtime are unknownFill the gap with a full rescanUSN change journalThe record stays on the volume sideRead back from the last USN

Figure 15: Once the requirements include not being able to run at all times, or picking up changes made during downtime, the change journal becomes an option.

7. Conclusion

FileSystemWatcher is no substitute for completion notifications. The truth is not in the event stream but in what is visible on disk right now. Make completion explicit via temp -> close -> rename / replace or done files / manifests, and decide ownership by taking a claim atomically. That is where the heart of the design lives.

Processing immediately on Created, trusting the count or order of Changed, treating “no more Changed” as completion, taking comfort in InternalBufferSize alone, seeing Error and never recovering - all designs to avoid. Instead, fold notifications into rescan requests, perform full rescans on startup / overflow / reconnection, take ownership through claim renames, and absorb duplicates and re-scans with idempotency.

In other words, the trick with FileSystemWatcher is to never equate “having received an event” with “being allowed to process.” Just separating those two greatly reduces the kind of monitoring code that breaks only once in a while.

8. References

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Frequently Asked Questions

Common questions about the topic of this article.

Is it safe to read a file in the FileSystemWatcher Created event?
No. Created only means the name became visible; it does not guarantee that the file is ready to read. During copies and transfers, Created fires the moment the file is created, and one or more Changed events can follow. The basic rule is that the sender declares completion explicitly with temp -> close -> rename/replace or with a done/manifest file, and the receiver looks only at the final name or the done file.
Can FileSystemWatcher drop notifications?
Yes. When the internal buffer (8192 bytes by default, never below 4096 bytes, capped at 64 KB) overflows, individual notifications are lost and the Error event fires. Once an overflow happens, the completeness of the individual event stream itself becomes questionable, so the safe move is a full rescan of the directory to re-examine the whole picture. A full rescan belongs at startup, on receiving Error, right after recreating the watcher, and periodically as insurance.
Why does Changed arrive so many times?
Even ordinary operations such as moving or saving can show up as several separate events, and on top of that you pick up whatever antivirus software or indexers touch. A design that trusts the count or the order is dangerous. Fold every notification into one kind of signal - a rescan request - concentrate scanning in a single worker, and during bursts coalesce for around 100-300 ms before scanning once. That is what stays stable.
Does increasing InternalBufferSize fix missed events?
No. Even raised to 64 KB, a notification burst beyond that still drops events, and it does nothing at all for the question of whether a notification means completion. The buffer uses non-paged memory, so enlarging it is not as casual as it sounds. The right order is to first narrow the watch scope with Filter and NotifyFilter, review IncludeSubdirectories, keep the event handlers lightweight, and add full rescans and idempotency.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.

Contact
Back to the Blog