KomuraSoft LLC
Contact
Menu

The Win32 Thread Pool API — Concurrency Without Creating Threads, via CreateThreadpoolWork

· · Windows, Multithreading, C++, Windows Development, Win32 API, Performance Improvement

“One CreateThread per client.” “One for the timer.” “One for waiting on an event.” — In native Windows code, threads tend to proliferate this way. Each thread consumes a stack and a kernel object, and creating and destroying them has a cost as well. The work is fine-grained, yet the thread is heavy — the thread pool is what the OS provides to absorb that mismatch.

The convenience of .NET’s ThreadPool and Task.Run is well known, but in fact Win32 native also has a well-designed, OS-standard thread-pool API. Comprehensively redesigned in Windows Vista, this API is the foundation of native concurrency: it can handle work, timers, waits, and asynchronous I/O through a unified callback mechanism. Aimed at developers writing Windows apps, services, and DLLs in C/C++, this article explains the structure and usage of this API, and the pitfalls that are easy to fall into, based on primary sources.

1. The Bottom Line First

  • For issuing a large number of short-lived jobs, and for replacing waiter-only threads, a thread pool beats your own CreateThread. You leave thread management to the OS and can reduce the thread count and context switches.1
  • What you should use is the new API (the CreateThreadpoolWork family). The Vista redesign made it simpler, more reliable, and higher-performance than the old API (the QueueUserWorkItem family), and you can also create several independent pools in one process.12
  • There are four kinds of object. work, which you submit jobs to; timer, which fires at a time or period; wait, which fires when a kernel object is signalled; and io, which fires when asynchronous I/O completes. All of them ride the same callback mechanism.3
  • Shutdown is “wait, then close”. A discipline of not leaving a running callback behind — waiting for completion with the WaitForThreadpoolWorkCallbacks family, or handling it in bulk through a cleanup group — is required.4
  • Inside a callback: do not block for a long time (if you will, CallbackMayRunLong); do not synchronously wait for completion on the same pool; do not dirty the thread’s state. These three are iron rules.56
  • Use from a DLL: watch for an unload race. Wait for completion in an explicit shutdown function, and know the dedicated APIs such as FreeLibraryWhenCallbackReturns.3

2. Why a Pool, and When a Pool

The idea of a thread pool is simple. Rather than creating a thread per job, you throw jobs (callbacks) at a group of worker threads the OS manages. The workers execute jobs one after another, and the OS adjusts the count to the load.

The official documentation lists concrete types of app where a pool pays off.1

  • Apps that issue a large number of small work items in parallel (search, network I/O, and so on)
  • Apps that frequently create and tear down short-lived threads
  • Apps that process independent work in parallel in the background
  • Apps that hold threads dedicated to waiting on kernel objects or events

The last item is easy to miss. If you have five threads that exist only to sleep so they can “run when the event is signalled”, those can be replaced by five wait objects on the pool, and the waiting is aggregated onto the pool’s waiter threads.

Conversely, there is also work that does not suit a pool. Work that needs a change of thread priority, that requires COM STA, that keeps running for the whole lifetime of the process — work that needs a “personality” on the thread is held on a dedicated thread. A worker thread is a shared resource; it is borrowed.

Choosing between a dedicated thread and a poolFirst check whether a thread personality such as priority or STA is needed, and whether it runs for a long time; only short-lived, high-volume, or wait-style work that matches neither goes on the thread poolYesNoYesNoNeed a personality such as priority or STA?Keep it on a dedicated threadDoes it run for a long time?Put it on the thread poolShort-lived work, waits, timers, I/O completions

Figure 1: The only work you may put on a pool is work that “needs no personality and finishes short”. Everything else stays on a dedicated thread as before.

There is also one piece of history to pin down. The thread-pool API has two generations. The old API that has continued since Windows 2000 (QueueUserWorkItem, RegisterWaitForSingleObject, and so on), and the new API comprehensively redesigned in Vista (the CreateThreadpoolWork family). The new API unifies the kinds of worker thread, provides dedicated persistent threads, multiple pools in one process, cleanup groups, and more, and official documentation states in so many words that it is “simpler, more reliable, better performing, and more flexible”.1 The old API also has structural constraints such as “you cannot cancel work once it is queued”.2 From here on, this article treats only the new API.

Correspondence between the old thread-pool API and the new APIThe old API's QueueUserWorkItem maps to the new API's work object, timer queues to timer, registered waits to wait, and BindIoCompletionCallback to ioQueueUserWorkItemworkTimer queuestimerRegistered waitswaitBindIoCompletionCallbackio

Figure 2: The migration target from the old API is one-to-one. An inventory of existing code can start from this correspondence.

3. The Four Objects — work, timer, wait, and io

At the centre of the new API are four kinds of object whose callback firing conditions differ.3

Object Creation function When the callback fires
work CreateThreadpoolWork When it is submitted with SubmitThreadpoolWork
timer CreateThreadpoolTimer When the specified time or period arrives
wait CreateThreadpoolWait When a kernel object becomes signalled
io CreateThreadpoolIo When asynchronous I/O on the associated handle completes
The thread pool's four objects and the callback mechanismwork fires on an explicit submit, timer on time, wait on a kernel-object signal, and io on asynchronous I/O completion; all of them run as callbacks on the same group of worker threadsWhich object?work（on submit）Timer, wait, or io?timer（time / period）Wait or io?wait（on signal）io（I/O completion）Workers run callback

Figure 3: The firing conditions differ, but all four are unified in a mechanism where “workers on the same pool run the callback”.

This unification is a practical strength. Instead of writing periodic processing, event response, and I/O-completion processing each on a dedicated thread, you can align them on one callback style. Timers are gathered into a single timer queue for the whole pool, and waits are aggregated onto a small number of waiter threads — threads that “only sleep” disappear from the process.1

Replacing waiter-only threads with wait objectsWaiter-only threads that used to sleep one per event become wait objects and are aggregated onto the pool's waiter threads, so the callback runs only when signalled5 dedicated waiter threads sleeping individuallyConsumes 5 stacks and 5 threads5 wait objectsAggregated onto the pool's waiter threadsThe callback runs only when signalled

Figure 4: Threads that “only sleep and wait” can be removed by turning them into wait objects. This is a clear first move for a pool migration.

4. The Basic Pattern — One Round Trip with a work Object

We walk through the manners once with the work object, which is the most frequently used.4

VOID CALLBACK WorkCallback(PTP_CALLBACK_INSTANCE instance,
                           PVOID context, PTP_WORK work)
{
    // The context is fixed at creation time. Per-item data is passed through a synchronised queue
    WORK_QUEUE* queue = (WORK_QUEUE*)context;
    ITEM* item = Dequeue(queue);        // Take one item under exclusive control
    ProcessItem(item);
}

// 1) Create (bind the callback to the shared context = the queue)
PTP_WORK work = CreateThreadpoolWork(WorkCallback, &queue, NULL);
if (!work) { /* Failure handling with GetLastError */ }

// 2) Submit once for each item you enqueue (keep the item count and the submit count in step)
Enqueue(&queue, item);
SubmitThreadpoolWork(work);

// 3) Stop the submitter, then wait for completion (TRUE also attempts to cancel work that has not yet started)
WaitForThreadpoolWorkCallbacks(work, FALSE);

// 4) Close
CloseThreadpoolWork(work);

There are two points to hold. First, you may SubmitThreadpoolWork the same work object more than once. Each submit runs the callback (in parallel).7 However, the context passed to the callback is fixed at creation time, so when you write “N items of the same kind of work” with one work object, you put a synchronised queue in the context as in the code above and take one item per submit (a design that creates a work object per item is also fine). Second, always wait for completion before you close. Closing the object while a running or queued callback remains, or freeing memory the callback refers to, is use-after-free as-is. To make this wait a safe close, stopping the submitter first is a prerequisite — in a structure where another thread can still Submit in parallel with the wait, a submit after the wait races with Close. Passing TRUE as the second argument of WaitForThreadpoolWorkCallbacks also attempts to cancel submits that have not yet started.

Lifecycle of a work objectCreate with CreateThreadpoolWork; submit with SubmitThreadpoolWork and callbacks run in parallel. At shutdown, first stop new submits, wait for every callback to complete with WaitForThreadpoolWorkCallbacks, then close with CloseThreadpoolWorkCreate with CreateThreadpoolWorkSubmit with SubmitThreadpoolWork (may be repeated)Callbacks run in parallelStop new submitsWait for completion with WaitForThreadpoolWorkCallbacksClose with CloseThreadpoolWork

Figure 5: The shutdown order is “stop submits → wait for completion → close”. Skip any of them and you get use-after-free or a race.

By default, callbacks run on the process-default pool. For many uses that is enough. The next chapter is for when you want to split pools.

5. Custom Pools and Cleanup Groups

Split the pool. You can create an independent pool with CreateThreadpool and set the upper and lower bounds on the thread count with SetThreadpoolThreadMaximum / SetThreadpoolThreadMinimum.8 The typical use is isolation. So that “batch-like work that may be slow” does not eat the workers of “work that needs to respond immediately”, you split the pools and give each its own budget of threads.

Bind them with a callback environment. Which pool the work runs on is specified by initialising a TP_CALLBACK_ENVIRON (callback environment), pointing it at the pool with SetThreadpoolCallbackPool, and passing that as the third argument of CreateThreadpoolWork and the like.9

Fold them with a cleanup group. In a module that creates many objects, shutdown processing tends to become a recitation of “wait for all of them, close all of them”. If you create a group with CreateThreadpoolCleanupGroup and attach each object to it through the callback environment, a single CloseThreadpoolCleanupGroupMembers performs completion-wait and release for every member object together.34

Binding the configuration through a callback environmentThe callback environment points at a custom pool and a cleanup group; work and timer created with that environment run on that pool, and a bulk operation on the cleanup group gathers completion-wait and releaseCallback environment (TP_CALLBACK_ENVIRON)Custom pool (control the thread count)Cleanup groupPassed when creating work / timer / wait / ioWait for completion and release in one shot

Figure 6: A callback environment is the mechanism that injects “which pool it runs on, and who cleans up” at object-creation time.

6. Pitfalls — Discipline Inside a Callback

Almost every thread-pool bug comes from “doing as you please on a borrowed thread”.

Blocking for a long time. The pool adjusts its thread count on the assumption that callbacks return promptly. Doing long work or a long wait at the defaults delays the execution of other callbacks. A callback that may run long should declare “this will run long” with CallbackMayRunLong (the pool takes it as a hint to add a thread), or be sent off to a dedicated thread in the first place. Note that CallbackMayRunLong returns FALSE when it cannot prepare a worker for other callbacks. If you keep blocking without checking the return value you still clog the pool, so when it is FALSE, fall to the side that does not block — split the work, send it to a dedicated thread, and so on.5

Synchronously waiting for completion on the same pool. A shape where, inside callback A, you wait with WaitForThreadpoolWorkCallbacks or similar for completion of work B submitted to the same pool becomes a pool-starvation deadlock the moment every worker is “waiting for another worker”. Rewrite a dependency between jobs not as a wait but as a continuation that “submits the next from B’s completion callback”.

The structure of a pool-starvation deadlockIf every worker thread synchronously waits for completion of other work submitted to the same pool, there is no free worker left to run that work, and everyone waits foreverWorker 1: waiting for work X to completeWork X and Y waiting to runWorker 2: waiting for work Y to completeNo free worker left to run themEveryone waits forever (starvation deadlock)

Figure 7: If you synchronously wait for a worker from inside a worker, there is no one left to run the work being waited on.

Dirtying the thread’s state. A worker thread is reused for the next callback. A change of thread priority, COM initialisation state, a value left in TLS, a lock you forgot to leave — any of these becomes contamination of the next (unrelated) callback. “A function you throw at a pool must not depend on the personality of the thread” has been an official caution since the old-API era.6 There are dedicated mechanisms for cleanup; for example, LeaveCriticalSectionWhenCallbackReturns can ask the pool to “release this lock when this callback returns”.3

A race with DLL unload. If the DLL that contains the code is unloaded while a callback is running, you get an access violation. The basic form is to thoroughly wait for completion in the DLL’s shutdown function; FreeLibraryWhenCallbackReturns is provided for the situation “this callback is the last job, and when it finishes I want the DLL freed including myself”. This API, however, only “lets go of one reference when the running callback returns”; it does not prevent an unload before the callback has started. You use it as a pair: take a module reference of your own with GetModuleHandleEx before you submit, and have the callback let that reference go with this API.3 And you must not do this completion-wait inside DllMain — as stated in “DllMain and the Loader Lock”, waiting for another thread inside DllMain is a deadlock pattern.

Preparing for a race between DLL unload and a callbackAn unload of the DLL while a callback is running becomes an access violation, so the basic form is to wait for completion in an explicit shutdown function and then close; when the last callback itself frees the DLL, use FreeLibraryWhenCallbackReturnsUnload during callbackAccess violationWait, then closeSafe unloadIn a shutdown functionFreeLibraryWhenCallbackReturnslast callback frees DLLNot inside DllMain

Figure 8: The basic form is to do “wait, then close” in an explicit shutdown function. Waiting inside DllMain invites a different deadlock.

Exceptions and crashes inside submitted work. An unhandled exception on a worker thread takes the process with it. Apply a policy of catching exceptions comprehensively at the entrance of the callback and logging them, the same way you would for a dedicated thread’s thread function.

7. How This Relates to the Standard Library and .NET — Which Layer You Write At

Finally, we sort out how this sits with the other tools.

  • If C++ std::async / std::thread suffice, they are the first candidate. They are portable, the code is short, and even the semantics of future are settled by the standard.10
  • Reasons to use the Win32 thread pool directly are when you (1) want a unified callback mechanism that includes timer, wait, and io, (2) want pool splitting or thread-count control, or (3) do not want to hold your own threads inside a DLL or COM component.
  • On the .NET side, ThreadPool and Task play the same role, and I/O completion is tied to IOCP. This basement structure is explained in “IOCP and the .NET Thread Pool”.
Deciding which layer's tools to write withIf standard C++ async or thread suffice, use those; use the Win32 thread pool directly when you need integration of timers, waits, and I/O completions, pool splitting or thread-count control, or you do not want your own threads inside a DLL or COM componentYesNoIntegration of timer, wait, and ioPool splitting / count controlAvoiding your own threads inside a DLLDo the standard C++ tools suffice?std::async / std::threadWhat do you need?Win32 thread pool

Figure 9: When in doubt, start with the standard library; this API’s turn comes when a requirement it cannot express appears.

In other words, this API is the foundation of concurrency at the point you have decided to “write native”. A realistic clean-up is a staged one: as the migration target from a proliferation of your own CreateThread, start by introducing the work object, then replace waiter-only threads with wait and timer threads with timer.

8. Summary

  • For issuing a large number of short-lived jobs, and for cleaning up waiter-only and timer-only threads, the OS-standard thread pool rather than your own threads. What you use is the new API from Vista onward.
  • At the centre are the four objects work, timer, wait, and io. The firing conditions differ; they are unified on the same group of workers and the same callback style.
  • The manners are “create → submit → wait for completion → close”. Multiple submits run in parallel. A cleanup group can bulk the shutdown processing.
  • The three iron rules of a callback: do not block for a long time (if you will, CallbackMayRunLong); do not synchronously wait on the same pool; do not dirty the thread’s state.
  • Use from a DLL: watch for an unload race. Wait for completion in an explicit shutdown function; do not do it in DllMain.
  • Where standard C++ or .NET suffice, use those. This API’s turn is when you need integration of timer/wait/io or pool control.

The thread-pool API is, among Win32 APIs, on the newer and better-designed side. Once you have finished the shift from the idea of “creating a thread” to the idea of “throwing a callback”, concurrency in native code becomes considerably clearer to write.

KomuraSoft LLC handles migration design from native code whose threads have proliferated onto the thread pool, design reviews of concurrent processing in C++ apps and DLLs, and root-cause investigation of hangs and crashes caused by pool starvation or callbacks. You are welcome to consult us starting from an inventory of existing code.

References

  1. Microsoft Learn, Thread Pools. On a thread pool being a collection of worker threads that efficiently execute asynchronous callbacks on an app’s behalf; on the types of app it suits (issuing a large number of small work items in parallel, frequently creating and tearing down short-lived threads, processing independent work in parallel, exclusive waits on kernel objects, and so on); and on the comprehensive Vista redesign (unification of worker-thread kinds, a single timer queue, dedicated persistent threads, cleanup groups, multiple pools in one process, and the new API). ↩ ↩2 ↩3 ↩4 ↩5

  2. Microsoft Learn, Thread Pooling. On the structure of the older thread-pool APIs (QueueUserWorkItem, timer queues, registered waits, BindIoCompletionCallback); on there being no way to cancel work once it is queued; and on the new thread-pool API introduced in Vista being stated as simpler and superior in reliability, performance, and flexibility. ↩ ↩2

  3. Microsoft Learn, threadpoolapiset.h header. On the function list that includes the four object-creation functions CreateThreadpoolWork, CreateThreadpoolTimer, CreateThreadpoolWait, and CreateThreadpoolIo; cleanup groups (CreateThreadpoolCleanupGroup); and cleanup tied to callback completion (LeaveCriticalSectionWhenCallbackReturns, FreeLibraryWhenCallbackReturns, and so on). ↩ ↩2 ↩3 ↩4 ↩5 ↩6

  4. Microsoft Learn, Using the Thread Pool Functions. On the basic procedure of creating with CreateThreadpoolWork, submitting with SubmitThreadpoolWork, waiting for completion with WaitForThreadpoolWorkCallbacks, and closing with CloseThreadpoolWork; and on a configuration example that combines a custom pool with a callback environment and a cleanup group. ↩ ↩2 ↩3

  5. Microsoft Learn, CallbackMayRunLong function (threadpoolapiset.h). On notifying the pool that the current callback may run for a long time, so the pool can use that as material for deciding whether to secure a thread for other callbacks; and on considering a dedicated thread for a long-running callback where possible. ↩ ↩2

  6. Microsoft Learn, Thread Pooling. On work items submitted to a thread pool, and the functions they call, having to be thread-pool safe; on not assuming that the executing thread is a dedicated, persistent thread; and on avoiding use of TLS and asynchronous calls that require a persistent thread. ↩ ↩2

  7. Microsoft Learn, SubmitThreadpoolWork function (threadpoolapiset.h). On being able to submit the same work object more than once without waiting for a preceding callback to complete, so that callbacks run in parallel; and on the pool being able to adjust (throttle) the thread count for efficiency. ↩

  8. Microsoft Learn, SetThreadpoolThreadMaximum function (threadpoolapiset.h). On being able to set an upper bound on the worker-thread count for a pool created with CreateThreadpool (the lower bound is SetThreadpoolThreadMinimum). ↩

  9. Microsoft Learn, CreateThreadpoolWork function (threadpoolapiset.h). On creating a work object from a callback function and a context pointer; and on the third argument, TP_CALLBACK_ENVIRON, being able to specify the callback’s execution environment (the pool it belongs to, and so on), with NULL meaning it runs in the default environment. ↩

  10. Microsoft Learn, <future>. On asynchronous execution per task through std::async and future being provided as the standard library, so you can write concurrency without managing threads directly. ↩

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Frequently Asked Questions

Common questions about the topic of this article.

What is better about a thread pool compared with creating your own threads with CreateThread?
Efficiency when you have a large number of short-lived jobs to do, and a reduction in thread-management code. Creating and destroying a thread has a cost you cannot ignore, so an app that repeats "CreateThread for each job and destroy it when done", or that holds many threads that exist only to sleep waiting for an event, can reduce its thread count and context switches by moving to a pool. The official documentation also lists as pool candidates apps that issue a large number of small work items in parallel, apps that create many short-lived threads, and apps that have threads dedicated solely to waiting on kernel objects. Conversely, work that "needs a personality of its own on the thread" — a change of priority, COM STA, long-running dedicated processing — should still be held on a dedicated thread as before.
How is this different from the older thread-pool functions such as QueueUserWorkItem?
The thread pool was comprehensively redesigned in Windows Vista. The current threadpoolapiset-family APIs (CreateThreadpoolWork and so on) are the new API; QueueUserWorkItem, RegisterWaitForSingleObject, and the like are the old (legacy) API. The new API unifies the kinds of worker thread, lets you create several independent pools in one process, and provides mechanisms such as bulk release through a cleanup group and lock release or DLL unload tied to callback completion. Official documentation also says the new API is simpler and superior in reliability, performance, and flexibility. The old API has structural constraints as well, such as "there is no way to cancel work once it is queued", so use the new API in new code.
Are there things you must not do inside a callback?
There are three large ones. First, blocking for a long time or doing long work at the defaults. The pool adjusts its thread count on the assumption that callbacks finish promptly, so for work that will take a long time you either declare it with CallbackMayRunLong or use a dedicated thread. Second, synchronously waiting for completion of other work you submitted to the same pool. If every worker ends up "waiting for another worker", you get a pool-starvation deadlock. Third, depending on the personality of the thread. Worker threads are shared across callbacks, so returning with a changed thread priority or COM initialisation state, or leaving state in TLS, contaminates the next callback. For cleanup at the end (releasing a lock or unloading a DLL), dedicated mechanisms such as LeaveCriticalSectionWhenCallbackReturns and FreeLibraryWhenCallbackReturns are provided.
Are there points to watch when using the thread pool from a DLL?
The greatest danger is "the DLL is unloaded while a callback is still running". If the callback runs after unload, you get an access violation. The DLL side must, in its shutdown processing, reliably wait for completion of the callbacks it issued — with a wait function such as WaitForThreadpoolWorkCallbacks, or CloseThreadpoolCleanupGroupMembers on a cleanup group — and only then close the objects. Waiting for this inside DllMain, however, can deadlock through interaction with the loader lock, so the rule is to do it in an explicit shutdown function, not in DllMain. For the situation where the callback itself wants to free the DLL because "this work is the last", a dedicated API, FreeLibraryWhenCallbackReturns, is provided.
Now that we have C++ std::async and .NET's ThreadPool, are there still occasions to use this API directly?
There are. The criterion is "does the tool at that layer suffice". If the granularity of concurrency you need in C++ is covered by std::async or std::thread, the standard library is the first candidate from a portability standpoint as well. On the other hand, wanting to unify timers, kernel-object waits, and asynchronous I/O completions in one callback mechanism; wanting to split pools and control thread counts per kind of work; not wanting to hold your own threads inside a DLL or COM component — those requirements are what the Win32 thread pool covers. The relationship with .NET's ThreadPool and IOCP is covered in a related article, and as long as you are writing native, knowing this mechanism that sits at the layer below is not wasted.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.

Contact
Back to the Blog