The Depths of Windows I/O (Part 3) — I/O Completion Ports (IOCP) and the .NET Thread Pool: The Basement Under async/await

Last time (Part 2), we looked at how asynchronous I/O is issued and the four routes through which completion is received. At that point, the one we only named — as “the real answer for handling huge numbers of concurrent I/O operations with a small number of threads” — was the I/O completion port (IOCP).

Why can a web server handle thousands of simultaneous connections with a dozen-odd threads? Why can we flatly say that an async/await I/O wait “doesn’t consume a thread”? Why does the rest of an await sometimes run on the UI thread and sometimes on the thread pool? — the answers to all three questions trace back to a single design: IOCP. This is the instalment of the series that connects most directly to .NET developers.

This is Part 3 of the series “The Depths of Windows I/O”. The overall structure is laid out at the start of Part 1.

Because this is a long article, here is a map of which question gets answered where.

• Question 1: why thousands of simultaneous connections can be handled with a dozen-odd threads → answered in Section 3 (the IOCP design that unifies the completion queue with thread-count control).
• Question 2: why an I/O wait can be said to “not consume a thread” → answered in Sections 5.1–5.2 (the full picture of one await round trip, and the precise meaning of “no thread exists during the wait”).
• Question 3: what decides which thread an await’s continuation runs on → answered in Section 5.3 (the captured context, and what ConfigureAwait(false) really means).

1. The Bottom Line First

• IOCP unifies a “completion notification queue” with “thread-count control.” Completion packets are pushed onto the queue FIFO, and worker threads pull them off with GetQueuedCompletionStatus (Section 3).¹
• Threads are woken LIFO. The “warm” thread that was working most recently is the one that picks up the next packet, so as long as the queue stays non-empty, context switches barely happen at all (Section 3.3).¹
• The concurrency value caps the number of runnable threads. The recommended starting point is the CPU count (0 uses the processor count). If a running thread blocks, a waiting thread is woken to fill the hole (Section 3.4).¹²
• The port can also be used for your own notifications. PostQueuedCompletionStatus lets you push packets unrelated to any I/O, so work requests for workers and shutdown instructions can flow through the same queue (Section 4).³
• For new server implementations, the Windows thread pool API (CreateThreadpoolIo) is recommended over raw IOCP. Internally it’s still IOCP, but it takes thread management off your hands (Section 4).¹
• The .NET thread pool has two storeys — worker threads and I/O completion threads — and asynchronous I/O handles are bound to the pool’s (own) IOCP. No thread exists for the duration of an await I/O wait; only the post-completion continuation rides on a thread (Section 5).⁴⁵⁶
• Where a continuation goes is decided by the “captured context.” Await on a UI thread and the rest goes back to the UI thread; capture nothing and it continues on the thread pool (or the thread that completed it). ConfigureAwait(false) is an instruction to stop that capture, not a guarantee of moving to the thread pool (Section 5.3).⁶

2. Where the “Brute-Force It With Threads” Design Breaks Down

First, let’s confirm the problem IOCP set out to solve. A naive server can be written as “one thread per connection.” Read synchronously, process, write. It’s an easy design to understand, but as connections grow it runs into two walls.

flowchart TB
    subgraph A["One thread per connection - synchronous I/O"]
        T1["Thread 1 - waiting on a read for connection 1"]
        T2["Thread 2 - waiting on a read for connection 2"]
        T3["Thread 3 - waiting on a read for connection 3"]
        TN["...threads keep growing with the connection count<br/>most are just sleeping, waiting on I/O"]
    end
    subgraph B["The IOCP model - asynchronous I/O"]
        Q["Completion queue<br/>gathers completion notifications from every connection"]
        W1["Worker 1"]
        W2["Worker 2"]
        WN["A small number of workers - roughly the CPU count"]
        Q --> W1
        Q --> W2
        Q --> WN
    end

Figure 1: On the left, threads grow in proportion to the number of connections. On the right, only “things that happened” are processed, by a small number of threads.

1. Threads aren’t free. Each one consumes a stack (1 MB reserved by default) and a kernel object, and as the count grows, the burden on the scheduler and on context switching piles up. Thousands of connections meaning thousands of threads is expensive even when most of them are simply “asleep, waiting for something to read.”
2. “How many you’d like to run” stops mattering once it exceeds the CPU count. Physically, only as many threads as there are CPUs can run at once. Making more threads than that runnable just increases the cost of switching between them.

Through Part 2 we saw ways to receive I/O completion notifications via “events” or “APCs.” But the event approach runs into the 64-object limit of WaitForMultipleObjects and gets fiddly to design waits around, and APCs are tied to the issuing thread. IOCP is the design built from the ground up around the shape of many I/O operations × a small number of threads.¹

3. The IOCP Design — Unifying the Queue and Thread Control

3.1. The Two Faces of CreateIoCompletionPort

CreateIoCompletionPort does two jobs despite its name: creating a new port, and associating a handle with an existing port.²

flowchart LR
    subgraph SRC["Associated handles - any number"]
        H1["File"]
        H2["Socket"]
        H3["Named pipe"]
    end
    subgraph PORT["I/O completion port"]
        Q["Completion packet queue - FIFO<br/>packet = bytes transferred +<br/>CompletionKey + OVERLAPPED pointer"]
        C["Concurrency control<br/>runnable thread count <= limit"]
    end
    subgraph W["Worker threads"]
        G1["Waiting with GetQueuedCompletionStatus"]
        G2["Waiting with GetQueuedCompletionStatus"]
    end
    H1 --> Q
    H2 --> Q
    H3 --> Q
    Q --> G1
    Q --> G2
    C -.controls.-> W

Figure 2: The structure of IOCP. Completions from many handles collect into one queue, and even the number of threads pulling from it is controlled.

The CompletionKey passed at association time is a free-form value used to tell the worker “this completion came from this handle” (the convention is to put a pointer to the connection object in it). A completion packet arrives carrying the CompletionKey, the OVERLAPPED pointer for that operation, and the number of bytes transferred. Which connection (CompletionKey), which operation (OVERLAPPED), and how far it got (byte count) — this is where the “operation’s docket” from Part 2 gets picked back up.²⁷

The target isn’t limited to “files.” Sockets, named pipes, mailslots, and so on — any handle that can speak overlapped I/O can be associated.¹ The “everything looks like a file” design we saw in Part 1 is at work here too.

3.2. The Journey of a Completion Packet

sequenceDiagram
    participant DRV as Kernel - IRP completion
    participant Q as Port queue - FIFO
    participant W as Worker thread

    Note over W: Waiting with GetQueuedCompletionStatus
    DRV->>Q: Push a completion packet<br/>byte count / CompletionKey / OVERLAPPED
    Q->>W: Wake one waiting thread and hand it over
    Note over W: Look at the packet and run completion handling<br/>run the continuation, issue the next I/O, etc.
    W->>Q: Finish processing and call GetQueuedCompletionStatus again
    Note over Q: If a packet is still waiting in the queue<br/>hand it over without waiting

Figure 3: Completion packets are pushed FIFO, and a worker loops between pulling one off and processing it.

When an asynchronous I/O completes, a completion packet is pushed onto the port’s queue in FIFO order. A worker calls GetQueuedCompletionStatus to receive one packet, and calls it again once processing is done — this loop is the skeleton of IOCP programming.¹⁷ There’s also GetQueuedCompletionStatusEx, which pulls several packets at once, cutting down the number of calls in high-frequency I/O.⁸

Here’s a good moment to squash the classic worker-loop bug. If GetQueuedCompletionStatus returns FALSE but the OVERLAPPED pointer comes back non-NULL, that means “you successfully pulled the completion packet for a failed I/O operation.”⁷ A failed operation still needs cleanup (error handling, plus releasing the docket and buffer from Part 2), so this packet must be processed. The only case where you can say “the packet itself couldn’t be retrieved” (a timeout, the port being closed, and so on) is when OVERLAPPED comes back NULL. Write the lazy if (!GetQueuedCompletionStatus(...)) break; and every failed I/O leaks straight through the cracks.

Here’s the skeleton in a form you can copy out. The order of checks maps directly onto the explanation above.

/* Skeleton of an IOCP worker loop (C / Win32) */
for (;;) {
    DWORD        bytes = 0;
    ULONG_PTR    key   = 0;
    OVERLAPPED  *ov    = NULL;

    BOOL ok = GetQueuedCompletionStatus(port, &bytes, &key, &ov, INFINITE);

    if (!ok && ov == NULL) {
        /* No packet was retrieved (e.g. the port was closed). The only condition allowed to break out */
        break;
    }
    if (!ok) {
        /* ov != NULL -> retrieved the completion packet for a "failed I/O".
           Cleanup (error handling, releasing the docket and buffer) is still required, so process it, don't break */
        DWORD err = GetLastError();
        handle_failed_io(key, ov, err);
        continue;
    }
    if (key == SHUTDOWN_KEY) {
        /* A shutdown packet pushed with PostQueuedCompletionStatus (Section 4) */
        break;
    }
    handle_completed_io(key, ov, bytes);   /* Normal completion handling. Keep it short (Section 3.4) */
}

The crux is not to dispose of ok == FALSE with a single branch, but to split it in two based on whether ov is NULL. Even with a timeout set (anything other than INFINITE), the check is the same — a timeout shows up as ok == FALSE with ov == NULL.

Note that when a thread calls GetQueuedCompletionStatus for the first time, that thread becomes associated with that port (a single thread can be associated with only one port at a time).¹ The mental picture of “a dedicated team of workers assigned to the port” is accurate here.

3.3. Threads Are Woken LIFO

This is where IOCP’s design gets clever. Packets are pushed FIFO, but the threads waiting for them are woken LIFO. In other words, the thread that was working most recently is the one that picks up the next packet too.¹

flowchart TB
    Q["Queue: P1 -> P2 -> P3, FIFO"]
    subgraph TH["Waiting threads - LIFO stack"]
        A["Thread A - ran until just now, warm"]
        B["Thread B - has been asleep a while"]
        C["Thread C - has been asleep for ages"]
    end
    Q -->|"P1, P2, and P3 all go<br/>to Thread A first if it's free"| A
    B -.->|"Only if A is already busy"| Q
    C -.->|"Rarely wakes at all"| Q

Figure 4: LIFO release. The busier things get, the more the same thread keeps cycling, while idle threads stay asleep.

This design has two benefits.

• Context switches don’t happen. As long as packets remain in the queue, a thread that finishes processing and calls GetQueuedCompletionStatus receives the next packet immediately, without waiting, and keeps running. The documentation explicitly states that, in a concurrency-value-1 scenario, “no thread switching occurs.”¹
• The cache stays warm. Because the same thread keeps cycling, its stack and scheduling-related state are more likely to still be sitting in the CPU cache. Sleeping threads are kept cheaply on standby for peak load.

3.4. The Concurrency Value — Counting “Runnable”

The concurrency value is the NumberOfConcurrentThreads passed when the port is created. It caps the number of runnable threads associated with that port, and while that cap is reached, no additional thread can pick up a packet.¹ Passing 0 uses the number of processors in the system, and the documentation states that the best overall maximum is the CPU count.²¹

The clever part is that this number counts not “awake threads” but “runnable” threads.

flowchart TB
    P["A packet arrives at the queue"]
    Q{"Is the number of runnable threads<br/>below the concurrency value?"}
    RUN["Wake a waiting thread and let it process"]
    HOLD["Leave it in the queue, wake no one<br/>a running thread will come pick it up"]
    BLK["A running thread entered<br/>a wait state for some other reason"]
    COMP["Wake enough waiting threads<br/>to make up for the drop in runnable count"]
    P --> Q
    Q -->|"below"| RUN
    Q -->|"at the limit"| HOLD
    BLK --> COMP

Figure 5: Concurrency control. Because the cap is on the “runnable count,” if someone blocks, the system automatically replenishes.

If a running worker enters some kind of wait (a lock, a page fault, or an accidentally synchronous I/O call), the runnable count drops, so the system wakes a waiting thread and fills the hole.¹ That’s why the convention is not to create exactly “the CPU count” of workers, but to keep more threads on standby than the concurrency value. If your workload mixes in long computation, you might also choose to raise the concurrency value itself, and the documentation’s position is that you ultimately tune it through profiling.¹

That said, replenishment isn’t magic. If a blocked thread later wakes up, the runnable count momentarily exceeds the limit (the documentation itself mentions this overshoot).¹ Keeping completion handling short is the golden rule, and it plays out in exactly the same shape in .NET in Section 6.

4. The Toolbox — the APIs That Support a Port

• PostQueuedCompletionStatus — pushes your own completion packet onto the queue without issuing any I/O.³ Handing out work to workers, shutdown instructions (pushing as many termination packets — colloquially “poison-pill” packets — as there are workers), notifications from other threads: being able to process I/O completions and your own messages in the same queue, the same loop greatly simplifies the design.
• GetQueuedCompletionStatusEx — pulls several completion packets at once. Effective for high-frequency I/O, where the overhead of one call per packet starts to bite.⁸
• SetFileCompletionNotificationModes — for the case from Section 5 of Part 2, “issued asynchronously but completed synchronously,” lets you choose a mode that doesn’t push a packet to the port (FILE_SKIP_COMPLETION_PORT_ON_SUCCESS). If a synchronous completion’s result is already known on the spot, routing back through the queue is just waste — this is that optimisation.⁹
• The Windows thread pool API — CreateThreadpoolIo / StartThreadpoolIo use IOCP internally while taking thread creation and management off your hands. Microsoft recommends that new server applications consider this first, and reach for raw IOCP only when you need explicit control over the concurrency value or thread management.¹ And the .NET thread pool is precisely this “IOCP plus automated thread management” implemented as a .NET runtime feature.

Just three pitfalls worth calling out. (1) Don’t block for a long time inside a worker (the Section 3.4 replenishment only softens the degradation, it doesn’t prevent it). (2) Identifying a completion packet is a two-stage affair — CompletionKey (per handle) and OVERLAPPED (per operation) — and Part 2’s lifetime management of the “docket” (don’t free it until completion) is still the lifeline here. (3) Don’t close a handle while an incomplete I/O is still outstanding — the cleanup behaviour (Section 6 of Part 1) and the cancellation etiquette (Section 6 of Part 2) apply just as they did.

5. The .NET Thread Pool — Two Storeys Built on Top of IOCP

This is where we get to the real subject: “the basement under async/await.”

The .NET thread pool has two kinds of threads. Worker threads, which run Task.Run and continuations, and I/O completion threads, which receive the completion of asynchronous I/O. ThreadPool.GetAvailableThreads(out workerThreads, out completionPortThreads) returns two separate numbers precisely because the internals really do have two storeys.⁴

And on Windows, the thread pool holds its own I/O completion port. The current low-level API for associating an OS handle with this port is ThreadPoolBoundHandle.BindHandle; asynchronous I/O on a bound handle is handled together with NativeOverlapped (the .NET-side face of the very OVERLAPPED from Part 2). When a FileStream or Socket opens a handle in asynchronous mode, this kind of binding happens internally (the older ThreadPool.BindHandle still exists doing the same job, but this is the one to reach for in new code).⁵ Put together:

• Part 2’s “asynchronous-mode handle + OVERLAPPED” is the mechanism for issuing
• This article’s IOCP is the mechanism for receiving completion
• The .NET thread pool’s I/O completion threads are the worker team running the GetQueuedCompletionStatus loop

— and with that correspondence, the Win32 picture becomes the .NET picture without modification.

5.1. await ReadAsync — the Round Trip in Full

Part 2’s Figure 7 left “genuine asynchronous I/O” as an unopened box. This time we open it all the way.

sequenceDiagram
    participant U as Calling thread - e.g. UI thread
    participant K as Kernel - IRP issue to completion
    participant Q as Thread pool's IOCP
    participant IO as I/O completion thread
    participant C as Where the continuation runs

    U->>K: ReadAsync issues an asynchronous read<br/>carrying the equivalent of OVERLAPPED
    K-->>U: ERROR_IO_PENDING, returns immediately
    Note over U: await registers a continuation on the incomplete Task<br/>and gives up the thread - for a UI thread, back to the message loop
    Note over K: The device is at work<br/>no thread anywhere is waiting during this time
    K->>Q: Push a completion packet
    Q->>IO: Wake one thread LIFO and hand it over
    Note over IO: Determine the result - byte count, status<br/>complete the Task and schedule the continuation
    IO->>C: Throw it to the captured context<br/>back to the UI thread, or the thread pool if none was captured
    Note over C: The code after the await runs

Figure 6: The full round trip of an await. Threads are only at work during “issuing” and “after completion” — the waiting time itself uses zero threads.

5.2. The Precise Meaning of “an I/O Wait Doesn’t Consume a Thread”

What this diagram is meant to drive home is that, between issuing and completion, no thread exists anywhere in user mode or the kernel purely to wait for that completion. Microsoft’s own async explainer (Async in Depth) makes the same point about I/O-bound Tasks — “there is nowhere a thread exists purely to wait for the completion” — walking all the way down to device drivers and interrupts.⁶ There are moments inside the kernel where a driver hands part of the work off to a system worker thread. But that’s a short piece of work meant to push the request forward, not a thread that blocks and keeps waiting for completion — that’s the scope of what’s guaranteed here.

Put in terms of what we’ve built up since Part 1: an IRP stays resident in the device stack not as a thread but as a data structure (Part 1); issuing returns immediately with ERROR_IO_PENDING (Part 2); and completion arrives as a chain of events — interrupt, then completion packet (this article). The design manages to sustain the state of “waiting” without needing the expensive resource that a thread represents.

That’s why an application that uses async/await correctly can sustain “10,000 I/O operations in flight at once” with a dozen-odd threads. Turn that around, though, and this property belongs only to I/O-bound Tasks. CPU work wrapped in Task.Run naturally occupies a worker thread the whole time, and the “fake asynchrony” from Section 7 of Part 2 is likewise quietly putting a thread to sleep behind the scenes.

5.3. Where the Continuation Runs

The last arrow in Figure 6 — “where the continuation gets thrown” — follows a clear rule.⁶

flowchart TB
    A["The Task completes and we want to run the continuation"]
    Q1{"At the point of the await, was a<br/>SynchronizationContext or a<br/>non-default TaskScheduler captured?"}
    Q2{"Was ConfigureAwait(false)<br/>attached?"}
    UI["Throw it back to the captured target<br/>e.g. run on the UI thread's message loop<br/>or on that TaskScheduler"]
    TP["No obligation to return to a specific place<br/>continues synchronously on the completing thread<br/>or runs on a thread-pool thread"]
    A --> Q2
    Q2 -->|"yes"| TP
    Q2 -->|"no"| Q1
    Q1 -->|"yes - e.g. a WPF/WinForms UI thread"| UI
    Q1 -->|"no - e.g. console app, ASP.NET Core"| TP

Figure 7: Where a continuation ends up. The reason you can touch UI controls directly after an await is that it’s being thrown back to the captured context.

• When you await on a WPF or WinForms UI thread, the SynchronizationContext is captured, and the rest returns to the UI thread. That’s why touching a control right after an await doesn’t raise a cross-thread violation. The practical side of this design is covered in “WPF/WinForms async and the UI Thread on One Sheet”.
• It’s not only SynchronizationContext that gets captured — if you await while running on a non-default TaskScheduler, that scheduler is captured too. Where neither exists (console apps, ASP.NET Core, code already on the thread pool), there’s no obligation to return to a specific place, so the continuation either runs on the thread pool or simply carries straight on, synchronously, on the thread that completed the Task.
• ConfigureAwait(false) is an explicit statement that “you don’t need to return,” not a guarantee that “it always moves to the thread pool.” If you await a Task that’s already completed (which includes the synchronous completions we saw in Part 2), no wait occurs at all and execution simply continues on the current thread. See “A Practical Decision Table for C# async/await - Task.Run and ConfigureAwait” for how to choose between the two in library code.

That last point is widely misunderstood, so let’s lay out the code side by side. First, code written under the misconception that ConfigureAwait(false) is an instruction to move to the thread pool.

// Bad: written under the misconception that "adding ConfigureAwait(false) means everything after this runs on the thread pool"
private async void OnLoadClick(object sender, EventArgs e)
{
    string csv = await File.ReadAllTextAsync(path).ConfigureAwait(false);

    // Expected: we're on the thread pool here, so the UI won't freeze
    // Reality: if the Task was already complete at the point of the await, no wait occurs and
    //          execution stays on the UI thread -> this heavy work freezes the UI
    var rows = ParseHeavy(csv);

    // And if it did complete asynchronously, the continuation isn't necessarily the UI thread either
    resultLabel.Text = $"{rows.Count} rows";   // -> can throw a cross-thread exception
}

All ConfigureAwait(false) says is “you don’t need to return to the captured context.” It doesn’t specify “where it runs,” so execution might stay on the UI thread, or it might continue on an I/O completion thread or a thread-pool thread. This code is broken precisely because either is possible.

Splitting the intent apart looks like this.

// Good: separately instruct "return to the UI or not" and "where the heavy work runs"
private async void OnLoadClick(object sender, EventArgs e)
{
    // Let UI code keep the capture (the rest returns to the UI thread)
    string csv = await File.ReadAllTextAsync(path);

    // If you want CPU-bound work pushed to the thread pool, say so explicitly with Task.Run
    var rows = await Task.Run(() => ParseHeavy(csv));

    // Definitely the UI thread here. Safe to touch controls
    resultLabel.Text = $"{rows.Count} rows";
}

// In library code (code with no UI), state the opposite: "there's no need to return"
public async Task<string> ReadConfigAsync(string path)
{
    string text = await File.ReadAllTextAsync(path).ConfigureAwait(false);
    return text.Trim();   // doesn't depend on the caller's context
}

The criterion is simple. Keep the capture in code that touches the UI, and be explicit with Task.Run when you want to change where something runs. Think of ConfigureAwait(false) as a library-side tool for declaring “this works safely regardless of the caller’s context.”

5.4. What Actually Clogs Things Up — Thread-Pool Starvation

One last pattern for how this basement gets clogged. If you wait synchronously inside a continuation or a worker (synchronous I/O, Task.Result/Wait(), a long lock wait), that thread stays occupied. IOCP’s own replenishment (Section 3.4) kicks in instantly as long as spare waiting threads remain. But once the spares run out, you’re in territory where the thread pool only injects new threads slowly. The moment load hits, starvation — “there’s a continuation to run but no thread to run it on” — sets in, and the whole application bogs down.

There are two entry points for investigating. Look at the availability of worker and I/O completion threads with ThreadPool.GetAvailableThreads.⁴ And grasp the real state of the thread pool and any blocking with event tracing — the procedure is set out in “Pinpointing "Slow" with PerfView and dotnet-trace — A Practical Introduction to .NET Performance Investigation”. Prevention is simple: keep the async path async all the way through — never mix in sync-over-async — that’s really all there is to it.

6. Summary

• IOCP is a mechanism that unifies the completion queue (FIFO) with thread-count control. Completions from many handles gather into a single port and are processed in a GetQueuedCompletionStatus loop.¹⁷
• Threads are released LIFO, so the busier things get, the more the same thread keeps cycling, minimising both context switches and cache misses.¹
• The concurrency value caps the number of runnable threads, with the CPU count (via 0) as the starting point. If a running thread blocks, waiting threads replenish it, but keeping completion handling short remains the golden rule.¹²
• PostQueuedCompletionStatus lets your own packets flow through too. For new implementations, the Windows thread pool API (IOCP underneath) is the first choice.³¹
• The .NET thread pool has two storeys — worker threads plus I/O completion threads — and asynchronous handles bind to the pool’s own IOCP. No thread exists for an await I/O wait; only the post-completion continuation rides on a thread.⁴⁵⁶
• Where a continuation goes depends on the captured context — back to the UI thread, or continuing on the thread pool. ConfigureAwait(false) is an instruction to stop that capture.⁶
• What actually clogs things up is almost always thread-pool starvation caused by sync-over-async creeping in. Keep the async path async, all the way through.

Next up is Part 4, “The Cache Manager — When Does Your WriteFile Actually Reach Disk?“ Part 2 noted that “if it’s already in the cache, the completion is synchronous,” and the cache’s shadow has flickered into view a few more times in this article too. Next time we face that cache head-on — deferred writes, read-ahead, FILE_FLAG_NO_BUFFERING, and the conditions under which “data you thought you’d written vanishes on a power loss.”

Related Articles

• The Depths of Windows I/O (Part 1) — Every Read and Write Becomes an IRP: The Big Picture of the I/O System
• The Depths of Windows I/O (Part 2) — Synchronous and Asynchronous I/O: What OVERLAPPED Really Means
• A Practical Decision Table for C# async/await - Task.Run and ConfigureAwait
• WPF/WinForms async and the UI Thread on One Sheet
• The Misconception That TCP Lets You Receive in the Same Units You Send — Designing Reception Around a Byte Stream
• Pinpointing “Slow” with PerfView and dotnet-trace — A Practical Introduction to .NET Performance Investigation
• A Practical Guide to Getting as Close to Soft Real-Time as Possible on Ordinary Windows

Related Consulting Areas

KomuraSoft LLC handles the design of Windows applications and servers that deal with large numbers of simultaneous connections and concurrent I/O, and investigates the causes of performance problems such as “the thread pool is clogging up” or “we went asynchronous but it isn’t any faster.”

• Windows Application Development
• Bug Investigation and Root-Cause Analysis
• Soft Real-Time Windows Application Development
• Contact Us

References

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Frequently Asked Questions

Common questions about the topic of this article.

What is an I/O completion port (IOCP)?

It's a Windows kernel mechanism that funnels the completion notifications of many asynchronous I/O operations into a single queue, and, in the same stroke, controls how many threads may run concurrently to process that queue. You create a port with CreateIoCompletionPort, associate handles such as files or sockets with it, and every time an asynchronous I/O completes, a completion packet is pushed onto the port's FIFO queue. Worker threads pull packets off the queue with GetQueuedCompletionStatus and process them. The key point is that this is not merely a notification queue — it doubles as a scheduling mechanism that keeps the number of runnable threads at or below a concurrency value. It lets you handle huge numbers of concurrent I/O operations efficiently with a small number of threads, and it is the foundation under both Windows server implementations and the .NET thread pool.

What should IOCP's concurrency value (the level of concurrency) be set to?

Microsoft's documentation states that the best overall maximum value is the number of CPUs on the computer. Passing 0 for NumberOfConcurrentThreads in CreateIoCompletionPort uses the number of processors in the system, so 0 is a reasonable default if you're unsure. This value caps the number of runnable threads, not the number of waiting threads. If a running thread enters a wait state for any reason, the system wakes another waiting thread to fill the hole, so if your processing mixes in long computation or blocking, you can also choose a larger concurrency value to increase how many packets are handled at once. Ultimately, tuning it in combination with profiling is what's recommended.

Why can we say that an async/await I/O wait doesn't consume a thread?

Because there is nowhere a dedicated thread exists to babysit that operation between issuing the I/O and its completion. As we saw in Parts 1 and 2, an issued request flows down the device stack as an IRP, and the call returns immediately with ERROR_IO_PENDING. All await does at that point is register a continuation on the still-incomplete Task and give up the thread. While the device is doing its work as hardware, there is no thread anywhere in user mode or the kernel that is simply waiting. Once it completes, a completion packet is pushed onto the thread pool's IOCP, and only then does an I/O completion thread run briefly to schedule the registered continuation. In other words, a thread is used only at the moment of issuing and in the post-completion follow-up; the waiting time itself proceeds with zero threads.

Which thread does the rest of an await (the continuation) run on?

By default, the SynchronizationContext (or TaskScheduler) in effect at the point of the await is captured, and the continuation is thrown back onto it. If you await on a WPF or WinForms UI thread, the rest runs on the UI thread, which is why you can touch controls directly right after the await. When there's no context worth capturing — a console app, ASP.NET Core, code already running on the thread pool, and so on — the continuation either runs on a thread-pool thread or simply carries straight on, on the thread that completed the Task. Adding ConfigureAwait(false) stops the capture, but that isn't a guarantee that it "always moves to the thread pool" — it's an instruction that it "doesn't need to return to a specific place." If you await a Task that has already completed, no wait occurs at all, and execution continues synchronously on the current thread. ConfigureAwait(false) is recommended in library code to avoid unnecessary round trips to the UI thread and to cut off the seeds of context dependence and deadlocks.

What happens if you block for a long time inside an IOCP worker thread or a .NET I/O completion thread?

It doesn't break instantly, but performance degrades because you've stepped outside what the mechanism was designed for. IOCP replenishes by waking a waiting thread whenever a running thread enters a wait state, but each replenishment inflates the level of concurrency and increases context switches. If blocking becomes routine, packets pile up in the queue and completion processing as a whole is delayed. The same is true in .NET: waiting synchronously — say, on synchronous I/O or Task.Result — inside an I/O completion thread or a continuation invites thread-pool starvation. The rule is to keep completion handling and continuations short, and to hand off any heavy work elsewhere. You can observe the available worker threads and I/O completion threads with ThreadPool.GetAvailableThreads, which is useful when investigating a bottleneck.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.