KomuraSoft LLC
Contact
Menu

Shared Memory Pitfalls and Practical Best Practices

· Updated: · · Shared Memory, IPC, Concurrency, C++, C#, Windows Development

Revision history (1 updates, last updated Sep 1, 2026)

A log of the changes made to this article. Where a pre-update version was archived, it stays readable at a permanent DOI link.

Retranslated as a full translation of the Japanese original. The previous English version was an abridgement that carried only part of the source, so sections, tables, Mermaid diagrams, figure captions and FAQ entries were missing. All of them have been restored to match the Japanese original, and the technical claims are the same as in the Japanese version. Read the version before this update (DOI: 10.5281/zenodo.21614535)
First published
Cite this article(DOI: 10.5281/zenodo.21614534)

This article is archived on Zenodo. Below are both the DOI that always resolves to the latest version and the DOI pinned to the version you are reading.

Go Komura (2026). Shared Memory Pitfalls and Practical Best Practices. KomuraSoft LLC. https://doi.org/10.5281/zenodo.21614534 https://comcomponent.com/en/blog/2026/03/18/000-shared-memory-pitfalls-best-practices/

DOI (latest version)
10.5281/zenodo.21614534
DOI (this version)
10.5281/zenodo.22217172

Image frames, inspection results, time-series logs, market depth data, huge buffers. When you want to exchange large data at low latency within the same machine, shared memory looks very attractive.

The slightly dangerous part, though, is that shared memory arrives wearing the face of “fast IPC.” In reality, shared memory is IPC that reduces copies in exchange for pushing the responsibility for consistency back onto your application.

  • Fast
  • Flexible
  • But the protocol is yours to build
  • And when it fails, the symptoms are spectacular

That is roughly the four-piece set.

The two faces of shared memoryDiagram showing that shared memory arrives wearing the face of fast IPC, but is in reality an IPC that reduces copies in exchange for pushing the responsibility for consistency back onto the application.The face of fast IPCWhat it really isFewer copiesConsistency is the application's jobYou build the protocol; failures are spectacular

Figure 1: Shared memory arrives wearing the face of fast IPC and pushes the responsibility for consistency back onto the application.

In this article, with Windows file mappings and POSIX shm_open / mmap in mind, we sort out where shared memory trips you up in practice, and how to design so that less goes wrong. Whether you use C/C++ or C#’s MemoryMappedFile, the essentials are almost identical.1

Intended Readers and Assumptions

This is written for developers who are about to decide how to pass large data between processes on the same machine. The main audience is people who touch Windows file mappings or POSIX shm_open directly from C / C++, but anyone coming in through C#’s MemoryMappedFile hits the same pitfalls. The chapters on pitfalls and design guidance (chapters 5 and 6) are language-independent.

A working sample sits in 6.9, in both C (Windows / MSVC) and C#. For the POSIX side, chapter 7 gives only the correspondence between API names, in a table.

Terms to Know Up Front

A handful of terms recur throughout the article. Here they are up front, so the first occurrence does not trip you up.

Term Meaning
IPC (Inter-Process Communication) Communication between processes: any mechanism for exchanging data or signals with another process. Pipes, sockets, named pipes, and shared memory all count
coherent Multiple views onto the same underlying object show the same content at the same point in time. It does not mean that a reader always reads a consistent, fully-updated record
ABI (Application Binary Interface) The binary-level contract that executables keep with each other, rather than a source-code one. It covers type sizes, alignment, padding, and the order of struct fields
SPSC / MPSC / SPMC / MPMC Shorthand for the number of producers and consumers. S is single, M is multi, P is producer, C is consumer. SPSC means 1 writer and 1 reader. Expanded in 4.2
lock-free A design that makes progress with atomic operations alone, without taking a lock. The term names a progress guarantee, that at least one thread always moves forward, which is a different property from being fast
sentinel A special value reserved to mean invalid or end-of-data. For an offset, you might decide that UINT64_MAX is invalid
NUMA (Non-Uniform Memory Access) A layout where memory is not equally distant from every CPU. Touching memory on a far node makes the same code visibly slower

1. The Conclusion First (In One Breath)

Stated rather bluntly, but in a way that is useful in practice:

  • Shared memory is a mechanism that shows the same byte sequence to multiple processes; it is not synchronization itself23
  • What it is fast at is moving large data within the same machine. If all you have is small control messages, pipe / socket / named pipe / queue is very often easier
  • With shared memory, being visible and being safe to read are separate problems
  • Do not build your design on volatile. Atomicity, ordering, and waiting need to be considered separately45
  • Putting raw pointers, HANDLEs, file descriptors, std::string, std::vector, std::mutex in directly will, almost always, make you cry later
  • Data placed in shared memory is safer when pushed toward fixed-width integers + explicit layout + a versioned header
  • Just putting magic / version / size / state / generation / heartbeat in a leading header dramatically changes how easy incidents are to investigate
  • The hard parts of shared memory are not speed but initialization, lifetime, recovery, permissions, and ABI
  • On Windows the skeleton is CreateFileMapping / OpenFileMapping / MapViewOfFile; on POSIX it is shm_open / ftruncate / mmap63
  • The starting point least likely to break is an SPSC (single-producer single-consumer) ring buffer or a double buffer

In short: shared memory is fast, but use it carelessly and you catch the it-feels-like-it-synchronizes-itself disease. Avoiding that is the first battle.

The difference between being visible and being safe to readDiagram showing that shared memory is a mechanism for showing the same byte sequence to multiple processes rather than synchronization itself, so being visible and being safe to read are separate problems.Mechanism that shows the same byte sequenceNot synchronization itselfBeing visibleBeing safe to readDesign them as separate problems

Figure 2: With shared memory, being visible and being safe to read are separate problems.

Knowledge map for this article

Shared memory uses CreateFileMapping/MapViewOfFile on Windows and shm_open/mmap on POSIX to make the same physical pages visible from several processes, but it is not synchronization in itself, so reading and writing several fields without synchronization can lead to an accident where a reader picks up a state the writer is still in the middle of writing. That accident is prevented with a commit protocol based on an SPSC ring buffer or double buffering, process-local resources such as raw pointers and HANDLEs cannot be placed there as they are and have to be replaced with offset-based references, and separating the control plane from the data plane so that notifications go out over a separate channel is recommended. In addition, fixing the ABI, avoiding initialization races by having only the creator initialize, designing crash recovery around abandoned mutexes on Windows or robust mutexes on POSIX, arranging the namespace and permissions, and fixing the size for each generation are the practical points that lower the accident rate.

Shared memory pitfalls and best practicesDiagram showing that shared memory only shares a range of bytes and is not synchronization in itself, the pitfalls of reading a half-written state, initialization races, crash recovery, ABI mismatches, and false sharing, and how they relate to countermeasures such as separating the control plane from the data plane and using an SPSC ring buffer, double buffering, and offset-based referencesusesusesusesmay causepreventspreventsrecommended forrecommended forrecommended forincompatible withrequiresmay causepreventsrecommended forusesusesmay causeconfigured byrecommended forincompatible withincompatible withrecommended fornot recommended forusesShared MemoryCreateFileMapping / MapViewOfFileshm_open/ftruncate/mmapFile Mapping (Memory-Mapped File)Torn ReadSPSC Ring BufferDouble-Buffered Commit ProtocolControl plane / data plane separationFixed Shared Memory HeaderOffset-Based ReferencesProcess-Local ResourcesShared Memory ABI DesignInitialization RaceCreator-only initializationCrash recovery designAbandoned Mutex (Windows)Robust Mutex (POSIX)False Sharing and Cache-Line ContentionNamespace and Permissions (Global/Local)Fixed Size per GenerationShared Memory Across MachinesSeparate Waitable Notification ChannelBusy-Loop PollingNamed Pipe

In the diagram a solid line marks a relation that always holds and a dashed line marks a conditional one (the conditions are given per relation on the detail page). The full list of relations (24 in total, with evidence and certainty) and the definitions of the main concepts are collected on the knowledge map detail page (in Japanese). Data: JSON-LD / Turtle

2. What Shared Memory Shares — and What It Does Not

Roughly speaking, shared memory is a mechanism that maps the same physical pages into the virtual address spaces of multiple processes. Windows uses a file mapping object and views; POSIX mmaps a shared memory object.273

Two points matter here.

  1. What is shared is the byte content, not the virtual addresses themselves
  2. Being coherent and being synchronized are different things
Seeing the same physical pages through separate viewsDiagram showing that shared memory maps the same physical pages into the virtual address spaces of multiple processes, and that what is shared is the byte content rather than the virtual addresses themselves.View in process ASame physical pagesView in process BWhat is shared is the bytes, not the virtual addresses

Figure 3: What is shared is the bytes of the same physical pages, not the virtual addresses themselves.

The Windows documentation also says that views created from the same file mapping object are coherent at a given point in time. But that does not mean a reader can always read a consistent, fully-updated record.8

For example, even if the writer intends to write

  • length
  • then payload
  • then the ready flag

in that order, a reader that reads with no synchronization at all may see the new length combined with the old payload. Shared memory does not fix this for you automatically.

So what shared memory shares is bytes. What it does not share is meaning, ordering, completion notification, and recovery policy. All of that you have to design yourself.

What shared memory shares and what it does notDiagram showing that shared memory shares only bytes, while meaning, ordering, completion notification, and recovery policy are not shared and have to be designed by the application.Shared memoryShares bytesDoes not shareMeaning and orderingCompletion notification and recovery policyYou design these

Figure 4: Bytes are shared, but meaning, ordering, completion notification, and recovery policy are yours to design.

3. Where Shared Memory Fits — and Where It Does Not

Situation Fit Reason
Passing large frames or buffers within the same machine Good fit Easy to reduce the number of copies
High-frequency sensor values, images, audio, market depth, and the like Good fit Easy to aim for low latency and high throughput
Exchanging only small commands and responses Poor fit The synchronization cost of control is relatively heavy
Communicating with other machines Not a fit Shared memory fundamentally assumes a single host
Long-term coexistence of different languages and different versions Hard Requires ABI and versioning design
Persistence is also required Depends on the goal File-backed mappings are viable, but persistence and IPC responsibilities mix easily

In practice, the separation control over messaging, data payload over shared memory is very strong. For example:

  • The UI process notifies the worker process to use the next frame via an event / pipe / socket
  • The actual frame payload lives in shared memory

That configuration tends to be peaceful.

Control over messaging, data payload over shared memoryDiagram showing a split where the UI process notifies the worker process to use the next frame over an event, pipe, or socket, while the actual frame payload is passed through shared memory.notification (event / pipe / socket)writes the frame payloadreads the frame payloadUI processWorker processShared memory

Figure 5: Notifications travel over messaging, and only the frame payload sits in shared memory.

4. The Four Things to Decide First

When designing shared memory, the first four things to decide are these.

The four things to decide firstDiagram showing the four items to decide first when designing shared memory: separating the control plane from the data plane, the concurrency model, ownership and lifetime, and the ABI and versioning.Shared memory designPlane separationConcurrency modelOwnership and lifetimeABI and version

Figure 6: At the start of the design, settle separation, concurrency model, ownership and lifetime, and ABI.

4.1 Separate the Control Plane from the Data Plane

Decide up front what goes into shared memory.

  • data plane: images, audio, record sequences, bulk data
  • control plane: start, stop, errors, reconnection, reinitialization, notifications

Just separating these two makes the shared-memory side of the design considerably simpler.

4.2 Narrow the Concurrency Model

  • SPSC: 1 producer / 1 consumer
  • MPSC: many writers / 1 consumer
  • SPMC: 1 writer / many readers
  • MPMC: many writers / many readers

The difficulty rises in roughly that order. Going straight to MPMC is not something we would recommend. You end up facing mutual exclusion between writers and memory ordering at the same time, and the bugs that surface later are hard to reproduce in tests.

Difficulty of the concurrency modelsDiagram showing that difficulty rises roughly in the order SPSC, MPSC, SPMC, MPMC, and that going straight to MPMC means facing mutual exclusion and memory ordering at the same time.SPSC (1 writer, 1 reader)MPSC (many writers, 1 reader)SPMC (1 writer, many readers)MPMC (many writers, many readers)Mutual exclusion and memory ordering at the same time

Figure 7: Concurrency-model difficulty rises roughly in this order, from SPSC to MPMC.

4.3 Decide Ownership and Lifetime

  • Who creates it
  • Who initializes it
  • Who deletes it
  • Who recovers it when a participant dies midway

If this is vague, behavior changes with every startup order and every restart, and isolating the cause gets hard.

4.4 Decide the ABI and Versioning

  • Layout
  • Type sizes
  • Alignment
  • Reserved areas
  • Version / feature flags
  • Compatibility guarantees

Shared memory is not an API; it is an ABI (binary interface) problem. Get this wrong and you end up with the nasty kind of failure where source compatibility exists but things break only at runtime.

Shared memory is an ABI matterDiagram showing that shared memory is not an API but a binary-level contract, an ABI, so being careless here produces failures where source compatibility holds but the program breaks only at runtime.shared memoryAn ABI contract, not an APIBe careless hereSource compatible, broken at runtime

Figure 8: Shared memory is an ABI contract; careless design breaks at runtime even when the source still compiles.

5. Common Pitfalls

5.1 Not Synchronizing

This is the most common one.

“We are looking at the same memory, so if I write it, they can read it.”

Sometimes they can. But that does not mean they can read it at the right time, in the right units, in the right order.

On both Windows and POSIX, access to shared memory assumes a separate synchronization mechanism alongside it. The Windows documentation says that access to shared views should be coordinated with mutexes / semaphores / events.2 The POSIX material likewise says that access to shared memory requires synchronization.9

The if-I-write-it-they-can-read-it pitfallDiagram showing that even when a read succeeds because both sides look at the same memory, reading at the right time, in the right units, and in the right order is a separate guarantee, so access to shared memory assumes a synchronization mechanism alongside it.If I write it, they can read itSometimes they canRight time, units, and order are separateAssumes a synchronization mechanism alongsidemutex / semaphore / event and the like

Figure 9: Being able to read is not the same as reading in the right units and order, and synchronization is the prerequisite.

5.2 Trying to Fix It with volatile

volatile is not a magic spell that rescues your shared-memory design. At minimum, atomicity and mutual exclusion are separate problems.45

For example, a design that places volatile bool ready; and busy-loops on it

  • wastes CPU
  • leaves the ordering guarantee between payload and ready vague
  • is not portable
  • easily picks up intermediate states

and pretty much nothing good comes of it.

On top of that, Windows’s WaitOnAddress is for threads within the same process. It is safer not to think of it as a cross-process waiting mechanism.10

Problems with watching a volatile flagDiagram showing that busy-looping on a volatile bool wastes CPU, leaves ordering guarantees vague, and easily picks up intermediate states, so it should not be the foundation of a design.Busy-loop on a volatile boolWastes CPUVague ordering guaranteesEasily picks up intermediate statesDo not build the design on thisWaitOnAddress is also for threads in one process

Figure 10: A volatile busy loop loses on all three counts: CPU, ordering, and intermediate states.

5.3 Letting Readers See Intermediate States

When shared memory fails, the symptoms look quite mundane.

  • Only the header is new
  • Only the payload is old
  • Only the length has been updated
  • A pair of two fields is inconsistent

Drawn out, the way the failure happens is simple. The reader just slips into the gap before the writer has finished writing length and payload.

reader processshared memorywriter processreader processshared memorywriter processlength is newpayload is still oldgrabs a half-finished statewhere only the header is newwrite 1024 into lengthread length1024read 1024 bytes of payloadcontents of the previous generationwrite payloadraise the ready flag

Figure 11: The reader slips into the gap before the writer finishes the payload and grabs an intermediate state.

That gap necessarily exists as long as writing length and writing payload are not a single indivisible operation. Atomically updating a single scalar is relatively simple, but publishing a record made of multiple fields requires a commit procedure.

Typically one of the following:

  • Protect the whole thing with a mutex
  • Use a double buffer and flip the currently valid buffer index at the end
  • Use a ring buffer with per-slot state / sequence
  • For 1 writer / many readers, take snapshots with a sequence counter

Even setting the ready flag last is still a half-finished design unless you decide with what memory ordering that flag is written and read. In shared memory, the moment of publication is itself the protocol.

5.4 Placing Pointers or Complex Objects As Is

This is another frequent pattern.

  • Raw pointers
  • HANDLE
  • File descriptors
  • std::string
  • std::vector
  • std::unordered_map
  • std::mutex
  • CRITICAL_SECTION

The pattern is placing these straight into shared memory and trying to use them from another process. In the reading process, the result is almost certainly an access violation or a meaningless value.

The reason is simple: virtual addresses and process-local resources only have meaning inside that process’s context. For Windows views too, mapping the same mapping in another process does not guarantee that the virtual addresses match.711

So if you need a reference, the basic approach is to hold it as an offset from the base address.

typedef struct ShmRef {
    uint64_t offset;   /* position relative to the start of the segment */
    uint32_t length;
    uint32_t kind;
} ShmRef;

This way, each process can resolve base + offset into its own address.

Reference by offset rather than by pointerDiagram showing that virtual addresses and process-local resources only have meaning inside their own process, so references are held as offsets from the base address and each process resolves base plus offset.insteadPlace a raw pointer or HANDLEMeaningless value in another processHold an offsetEach process resolves base + offset

Figure 12: Hold references as offsets from the base address, not as raw pointers.

5.5 Breaking the ABI

Shared memory is a binary contract, not source code. Which means every one of the following differences matters.

  • Sizes of int / long
  • Representation of bool
  • The underlying type of enums
  • Size of wchar_t
  • 32-bit / 64-bit differences
  • #pragma pack
  • Compiler / language differences
  • Alignment / padding
  • Little-endian / big-endian

Within a single host, endianness is usually consistent, but just adding ARM64 support or a mixed toolchain produces perfectly ordinary mismatches.

So for structures placed in shared memory, we strongly recommend:

  • Fixed-width integers such as uint32_t / uint64_t
  • Explicit padding / reserved fields
  • A header with version, header_size, record_size, total_size
  • static_assert(sizeof(...)) where needed
  • No non-trivial objects
What breaks the ABI and how to prevent itDiagram showing that differences in type sizes, alignment, pack and padding, and 32-bit versus 64-bit break the binary contract, and that fixed-width integers, explicit layout, and a versioned header prevent it.Differences in type sizesThe binary contract breaksDifferences in pack and padding32-bit / 64-bit differencesFixed-width integers + explicit layoutPut version and size in the header

Figure 13: Differences in type sizes and padding break the ABI, so lean on fixed-width integers and explicit layout.

5.6 Initialization Races

Shared memory breaks easily on the assumption that whoever created it must have initialized it.

On Windows, when CreateFileMapping hits an existing name it returns the existing object, and GetLastError() reports ERROR_ALREADY_EXISTS. The initial pages of a pagefile-backed mapping start out zeroed.8 On POSIX, a new shared memory object starts with length 0 and gets its size from ftruncate. Newly allocated bytes are zero-initialized. Creation with O_CREAT | O_EXCL is atomic.3

If you do not know these differences and you

  • use it immediately after opening
  • have no initialization-complete flag
  • let participants initialize concurrently
  • never look for a version mismatch

then it breaks depending on startup order.

At minimum, put these states in the leading header.

  • INITIALIZING
  • READY
  • BROKEN

And only the creator initializes; joiners wait for READY. This etiquette alone makes the world considerably quieter.

How to avoid an initialization raceDiagram showing that putting INITIALIZING, READY, and BROKEN states in the leading header, having only the creator initialize and raise READY, and having joiners wait for READY avoids initialization races.The creator creates itOnly the creator initializes itSet state to READYA joiner opens it but does not use it yetWait for READYStart using it

Figure 14: Only the creator initializes, and joiners wait for READY in the header before using the block.

5.7 Not Thinking About Crash Recovery

What happens when the writer dies in the middle of updating shared data? Ship to production with that undefined and the look on everyone’s face during an outage suddenly gets serious.

A Windows mutex becomes abandoned when the owning thread exits without releasing it, and waiters receive WAIT_ABANDONED. That means the shared resource may be in an indeterminate state.12 With POSIX robust mutexes as well, when the owner dies you get EOWNERDEAD back, and after repairing the state you call pthread_mutex_consistent().1314

What matters here is not to just keep going. Recovery requires at least one of the following:

  • A generation number
  • The last committed sequence
  • A heartbeat
  • A dirty / clean flag
  • Journal-style two-phase commit
  • A full reinitialization procedure for corruption
When the writer dies mid-updateDiagram showing that when an owner exits without releasing, Windows returns WAIT_ABANDONED and a POSIX robust mutex returns EOWNERDEAD, the shared resource may be in an indeterminate state, and the right move is to go to a recovery procedure rather than keep going.Writer dies mid-updateWAIT_ABANDONED (Windows)EOWNERDEAD (POSIX robust)The shared resource may be indeterminateDo not just keep goinggeneration / heartbeat / two-phase commit / reinitialize

Figure 15: When the owner dies, suspect an indeterminate state and move to the recovery procedure instead of continuing.

5.8 False Sharing and Cache-Line Contention

Shared memory is often said to be fast. But if hot counters are packed into the same cache line, the line bounces between CPUs and things slow down by an amount you cannot ignore.

The classic example is

  • the producer updates write_index
  • the consumer updates read_index
  • both sit on the same cache line

In that case,

  • split hot fields onto separate cache lines
  • separate frequently updated fields from rarely updated ones
  • aim for one writer per cache line

and that alone changes things considerably. You often hear about aligning to 64 bytes; treat that as 64 bytes being a common value on many CPUs rather than an absolute law.

The classic false sharing case and its fixDiagram showing that when the write_index updated by the producer and the read_index updated by the consumer sit on the same cache line, the line bounces between CPUs and slows things down, so hot fields are split onto separate cache lines.Producer updates write_indexSame cache lineConsumer updates read_indexThe line bounces between CPUs and slows downSplit hot fields onto separate cache lines

Figure 16: Hot counters on the same cache line slow things down, so move them to separate lines.

5.9 Taking Names, Permissions, and Security Lightly

Named shared memory is convenient, but careless names and permissions will come back to bite you.

On Windows,

  • there are Global\ and Local\ namespaces
  • creating a Global\ file mapping from outside session 0 requires SeCreateGlobalPrivilege
  • object names share a namespace with events / semaphores / mutexes / waitable timers / jobs

— those are the quirks you get.1582

In other words:

  • you name it "Global\\MyApp" and figure the service and the desktop app can share it
  • but it fails on permissions
  • and on top of that, a mutex with the same name was created first, so you get ERROR_INVALID_HANDLE

which is exactly the very Windows-flavored mess you would expect.

The mess around the Global namespaceDiagram showing that trying to share between a service and a desktop app through a Global name can fail on permissions for lack of SeCreateGlobalPrivilege, or collide with a same-named mutex that already exists because the namespace is shared.Want to share through a Global nameFails on permissionsCollides with a same-named mutexCreation requires SeCreateGlobalPrivilegeEnds in ERROR_INVALID_HANDLE

Figure 17: The Global namespace is where you step in two kinds of mess: permissions and name collisions.

On the POSIX side too, treating shm_open’s mode or umask lightly makes the object unnecessarily visible, or conversely impossible to open.3

Shared memory is not safe just because it is only memory. From any process with read permission, it is quite plainly visible. If you put confidential information in it, you need to think about paging / swap / dumps / permissions, exactly as you would for ordinary memory.

5.10 Resizing and Upgrading Carelessly

“I would like to grow the shared memory a bit later” is a fairly dangerous request.

  • A Windows mapping object has a size fixed at creation8
  • On POSIX too, unless you keep ftruncate and mmap consistent, the participants’ mapped lengths stop matching316

In practice, it is safer to make the size immutable within a generation. If you need to grow it,

  1. create a segment with a new version / name / generation
  2. switch the participants over
  3. close the old segment

— that route breaks far less often.

Grow the size by switching generationsDiagram showing that the size of a shared memory segment stays immutable within a generation, and that growing it by creating a new-generation segment, switching participants over, and closing the old segment leaves far less room for things to go wrong.Want to grow it laterCreate a new-generation segmentSwitch the participants overClose the old segmentResize in place is dangerous

Figure 18: Fix the size within a generation and grow by switching to a new-generation segment.

5.11 Cramming Even Notifications into Shared Memory

A common pattern is

  • write ready = 1 into shared memory
  • the peer does while (!ready) Sleep(1);

This works at first. But later it comes back as

  • wasted CPU
  • latency jitter from Sleep(1)
  • missed updates that are hard to notice
  • timeouts and shutdown notifications that are hard to write cleanly

Push shared memory toward the data side, and move notification out to primitives you can wait on.

  • Windows: event / semaphore / mutex / named pipe and the like217
  • POSIX: semaphores / process-shared mutex + condvar and the like1819
Move notification out to a primitive you can wait onDiagram showing that watching a shared memory flag with Sleep wastes CPU, adds latency jitter, and hides missed updates, so shared memory stays on the data side and notification moves to primitives you can wait on such as events and semaphores.Watch a ready flag with SleepWasted CPU and latency jitterMissed updates are hard to noticeNotification goes to a waitable primitiveWindows: event / semaphore and the likePOSIX: semaphore / condvar and the like

Figure 19: Receive notifications through a primitive you can wait on, not by watching a flag.

5.12 Thinking “This Lets Me Share Across Machines Too”

There is a moment when you are tempted to think: if I use a file-backed mapping and map a file on a network share, maybe I can do shared memory across machines too.

That is dangerous.

The documentation for Windows CreateFileMapping also says that coherence is not guaranteed for remote files. If two machines map the same page as writable, each sees only its own writes, and nothing is merged when the disk is updated.8

Shared memory is fundamentally a single-host mechanism. If you need to cross machines, choosing socket / RPC / message broker outright is far better for your sanity.

Do not share across machinesDiagram showing that mapping a file on a network share does not guarantee coherence for remote files, that shared memory is a single-host mechanism, and that crossing machines calls for sockets, RPC, or a message broker.Want to share by mapping a remote fileCoherence is not guaranteedShared memory is a single-host mechanismGo to socket / RPC / message broker

Figure 20: Shared memory is a single-host mechanism, so pick messaging when you cross machines.

6. Best Practices

6.1 Separate the Control Plane from the Data Plane

Taking the separation decided in 4.1 down to implementation-level assignments gives you this (for how it goes wrong, see 5.11).

  • Shared memory: frame, sample, batch, snapshot
  • Event / semaphore / pipe / socket: ready, consumed, stop, error, reconnect

This separation improves design clarity even before it improves performance.

6.2 Put a Fixed Header at the Front

At minimum, we strongly recommend a leading header like this.

typedef struct SharedHeader {
    uint32_t magic;
    uint16_t abi_version;
    uint16_t header_size;

    uint32_t state;          /* 0=initializing, 1=ready, 2=broken */
    uint32_t flags;

    uint64_t total_size;
    uint64_t generation;
    uint64_t heartbeat_ns;

    uint64_t payload_offset;
    uint64_t payload_size;

    uint64_t write_seq;
    uint64_t read_seq;

    uint8_t  reserved[64];
} SharedHeader;

The points are

  • magic rejects a foreign or uninitialized segment
  • abi_version and header_size reject layout differences
  • state rejects a segment that is still initializing
  • generation detects re-creation
  • heartbeat shows liveness
  • reserved leaves an escape hatch for future extension

What is painful about shared memory is that it is hard to see what is happening. That is exactly why you give it observability metadata from the start.

What each field of the fixed header doesDiagram showing the division of labor in the leading header, where magic rejects foreign or uninitialized segments, abi_version and header_size reject layout differences, state rejects segments still initializing, generation detects re-creation, and heartbeat shows liveness.Leading fixed headermagic rejects foreign segmentsversion rejects differencesstate rejects mid-initializationBecomes observability metadatageneration detects re-creationheartbeat shows liveness

Figure 21: Each field of the fixed header rejects something: a foreign segment, a layout difference, or an unfinished initialization.

6.3 Use Offset References

Hold references as offsets, not pointers.

  • Resolve them as base + offset
  • Add range checks on offset + length
  • Define a sentinel for invalid values

This alone substantially reduces address-mismatch failures.

6.4 Narrow the Concurrency Model

Of the four models in 4.2, the first choice should be one of these two.

  • SPSC ring buffer
  • 1-writer / many-reader snapshot

An SPSC ring buffer is just an array of fixed-length slots where the producer writes at the write_seq position and the consumer reads from the read_seq position. Because there is exactly one writer and one reader, both indices move in one direction only.

Ring buffer with 8 slotswraps to slot 0 at the endslot 0readslot 1readslot 2unreadslot 3unreadslot 4being writtenslot 5freeslot 6freeslot 7freeconsumerread_seq = 2advances after readingproducerwrite_seq = 4advances after writing

Figure 22: The structure of an SPSC ring buffer. Producer and consumer advance separate indices in one direction.

The key is never to break the order of write first, then advance the index and read first, then advance the index. And, as 5.8 says, put write_seq and read_seq on separate cache lines.

If you need multiple writers, things usually go better when you reduce the number of places responsible for consistency, for example:

  • only the enqueue is lock-free / atomic
  • actual data updates are funneled to a single consumer

6.5 Make the Commit Protocol Explicit

A design where you cannot explain in words from which moment it is safe to read is dangerous.

For a double buffer, for example, you settle on a publication ritual:

  1. write to the unpublished buffer
  2. finalize the checksum and length
  3. switch the active buffer index with release semantics
  4. the reader reads the active index with acquire semantics
  5. after reading, check that the index has not changed

Drawn out, it becomes clear that there is exactly one moment where the switch happens.

readerbuffer Bactive indexbuffer Awriterreaderbuffer Bactive indexbuffer Awriteractive index is Aactive index is Bdiscard what was readand read again from Bread with acquireAread buffer Awrite to the unpublished Bfinalize length and checksumswitch to B with releaserecheck the index after readingit had changed to B

Figure 23: The publication ritual of a double buffer. The reader rechecks the active index after it finishes reading.

Skip that step of rechecking the index after reading, and the writer will reuse the same buffer for its next write while the reader is still in it, producing the same half-finished state as 5.3. Note that with only two buffers the index can flip again during the re-read, so if updates are fast, add more buffers or move to the sequence-counter approach listed in 5.3.

6.6 Fix the Size per Generation

Rather than resizing in place, cutting generations like

  • name = MyShm.v3
  • abi_version = 3
  • generation = 42

is easier to maintain.

Shared memory does not type-check at call time the way an API does. That is why not breaking an ABI once it is decided matters so much.

6.7 Build In Observability

At minimum, having these around helps a lot.

  • Last update time
  • Last successful sequence
  • Drop count / overwrite count
  • Version mismatch count
  • Attach / detach count
  • Last error code
  • Heartbeat

When shared memory breaks, the logs are usually thin. Adding your own counters makes incident response considerably easier.

6.8 Write the Failure-Path Tests First

The happy path alone is not enough. At the very least, look at these.

  • Force-kill the writer mid-update
  • The reader lags and the ring overflows
  • Connecting with a version mismatch
  • Mixed 32-bit / 64-bit
  • Opening across sessions
  • Insufficient permissions
  • A predecessor process restarts while still holding an old generation
  • Cache misses / NUMA effects under continuous huge-data transfer

With shared memory, tests that break things are worth more than happy-path tests.

6.9 A Minimal Round-Trip Sample

Here is everything above boiled down to a minimal working configuration. It puts one fixed-layout block into a pagefile-backed Windows file mapping and exchanges notifications with two auto-reset events. That is all it does.

The four shared rules are these.

  • The block holds only fixed-width integers and fixed-length arrays. No pointers, no HANDLEs
  • magic / abi_version / block_size / state sit at the front
  • Finish writing the payload, then finalize the length, and only then raise the event
  • The event names are different from the file mapping name (on Windows, events, semaphores, mutexes, waitable timers, jobs, and file mappings share one namespace)815
The round trip of the minimal sampleDiagram showing the round trip in which the sender finishes initialization, writes the payload, finalizes the length, and raises the event, while the receiver waits for the event, checks the ABI and state, range-checks the length before reading, and replies in the same order.receivershared blocksenderreceivershared blocksenderinitialize and set state to READYfinish writing the payload, then finalize the lengthraise the request eventcheck magic / version / staterange-check the length and readwrite the reply payload, then finalize the lengthraise the reply event

Figure 24: The round trip of the minimal sample. Finish the payload, finalize the length, and only then notify.

The sender comes first.

/* shm_writer.c : the sender. Start this one first.
 *   cl /W4 /nologo shm_writer.c        (kernel32.lib is linked by default) */
#include <windows.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>

#define SHM_NAME  L"Local\\KsShmDemo.v1.Block"
#define EVT_REQ   L"Local\\KsShmDemo.v1.Request"
#define EVT_REP   L"Local\\KsShmDemo.v1.Reply"
#define SHM_MAGIC 0x314D4853u   /* 'S','H','M','1' laid out little-endian */
#define SHM_ABI   1u
#define STATE_INITIALIZING 0u
#define STATE_READY        1u

#pragma pack(push, 8)
typedef struct DemoBlock {
    uint32_t magic;
    uint32_t abi_version;
    uint32_t block_size;
    uint32_t state;
    uint32_t request_len;
    uint32_t reply_len;
    char     request[256];
    char     reply[256];
} DemoBlock;          /* 24 + 256 + 256 = 536 bytes */
#pragma pack(pop)

int main(void)
{
    HANDLE hMap = NULL, hReq = NULL, hRep = NULL;
    DemoBlock *blk = NULL;
    uint32_t len = 0;
    DWORD waited = 0;
    int rc = 1;

    /* 1. Create a pagefile-backed mapping. Its initial pages start out zeroed.
     *    If the same name already exists, CreateFileMappingW "succeeds and returns
     *    the existing object", so a NULL check alone cannot stop a second sender.
     *    Fall through to the memset below and it wipes the shared block of a peer
     *    that is still running. GetLastError() is set on success too, so read it
     *    immediately. */
    hMap = CreateFileMappingW(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE,
                              0, (DWORD)sizeof(DemoBlock), SHM_NAME);
    if (hMap == NULL) {
        printf("CreateFileMapping failed: %lu\n", GetLastError());
        goto cleanup;
    }
    if (GetLastError() == ERROR_ALREADY_EXISTS) {
        printf("%ls is already in use. Only one sender at a time\n", SHM_NAME);
        goto cleanup;
    }

    blk = (DemoBlock *)MapViewOfFile(hMap, FILE_MAP_ALL_ACCESS, 0, 0, sizeof(DemoBlock));
    if (blk == NULL) {
        printf("MapViewOfFile failed: %lu\n", GetLastError());
        goto cleanup;
    }

    /* 2. Notification events. Give them names different from the mapping */
    hReq = CreateEventW(NULL, FALSE, FALSE, EVT_REQ);   /* auto-reset / non-signaled */
    hRep = CreateEventW(NULL, FALSE, FALSE, EVT_REP);
    if (hReq == NULL || hRep == NULL) {
        printf("CreateEvent failed: %lu\n", GetLastError());
        goto cleanup;
    }

    /* 3. Only the creator initializes. state is raised last */
    memset(blk, 0, sizeof(*blk));
    blk->magic       = SHM_MAGIC;
    blk->abi_version = SHM_ABI;
    blk->block_size  = (uint32_t)sizeof(DemoBlock);
    blk->state       = STATE_INITIALIZING;
    MemoryBarrier();
    blk->state = STATE_READY;

    /* 4. Never break the order: payload -> barrier -> length -> notification */
    strcpy_s(blk->request, sizeof(blk->request), "ping from writer");
    MemoryBarrier();
    blk->request_len = (uint32_t)strlen(blk->request);
    if (!SetEvent(hReq)) {
        printf("SetEvent failed: %lu\n", GetLastError());
        goto cleanup;
    }

    /* 5. Wait for the reply. Never wait forever */
    waited = WaitForSingleObject(hRep, 5000);
    if (waited == WAIT_TIMEOUT) {
        printf("no response from the reader\n");
        goto cleanup;
    }
    if (waited != WAIT_OBJECT_0) {
        printf("WaitForSingleObject failed: %lu\n", GetLastError());
        goto cleanup;
    }

    /* 6. Range-check the length before reading */
    len = blk->reply_len;
    if (len > sizeof(blk->reply)) {
        printf("reply_len is out of range: %u\n", len);
        goto cleanup;
    }
    printf("reply: %.*s\n", (int)len, blk->reply);
    rc = 0;

cleanup:
    /* Close every view and handle and the name disappears with them.
       Do not exit before the reader */
    if (hRep != NULL) CloseHandle(hRep);
    if (hReq != NULL) CloseHandle(hReq);
    if (blk  != NULL) UnmapViewOfFile(blk);
    if (hMap != NULL) CloseHandle(hMap);
    return rc;
}

The receiver keeps everything from SHM_NAME through DemoBlock exactly identical to the sender and only swaps out main. In production you would factor that shared part into a header file.

/* shm_reader.c : the receiver. The constants and the DemoBlock definition are
 *   identical to shm_writer.c.
 *   cl /W4 /nologo shm_reader.c */
int main(void)
{
    HANDLE hMap = NULL, hReq = NULL, hRep = NULL;
    DemoBlock *blk = NULL;
    uint32_t len = 0;
    int rc = 1;

    hMap = OpenFileMappingW(FILE_MAP_ALL_ACCESS, FALSE, SHM_NAME);
    if (hMap == NULL) {
        printf("OpenFileMapping failed: %lu / is the writer running?\n", GetLastError());
        goto cleanup;
    }

    blk = (DemoBlock *)MapViewOfFile(hMap, FILE_MAP_ALL_ACCESS, 0, 0, sizeof(DemoBlock));
    if (blk == NULL) {
        printf("MapViewOfFile failed: %lu\n", GetLastError());
        goto cleanup;
    }

    hReq = CreateEventW(NULL, FALSE, FALSE, EVT_REQ);
    hRep = CreateEventW(NULL, FALSE, FALSE, EVT_REP);
    if (hReq == NULL || hRep == NULL) {
        printf("CreateEvent failed: %lu\n", GetLastError());
        goto cleanup;
    }

    /* 1. Wait for the notification first. The writer only raises it
     *    after it has finished initializing */
    if (WaitForSingleObject(hReq, 5000) != WAIT_OBJECT_0) {
        printf("no request arrived\n");
        goto cleanup;
    }

    /* 2. Check the ABI and the state before touching anything */
    if (blk->magic != SHM_MAGIC || blk->abi_version != SHM_ABI ||
        blk->block_size != (uint32_t)sizeof(DemoBlock)) {
        printf("ABI mismatch: magic=%08X abi=%u size=%u\n",
               blk->magic, blk->abi_version, blk->block_size);
        goto cleanup;
    }
    if (blk->state != STATE_READY) {
        printf("initialization is not finished yet: state=%u\n", blk->state);
        goto cleanup;
    }

    /* 3. Range-check the length, then read */
    len = blk->request_len;
    if (len > sizeof(blk->request)) {
        printf("request_len is out of range: %u\n", len);
        goto cleanup;
    }
    printf("request: %.*s\n", (int)len, blk->request);

    /* 4. payload -> barrier -> length -> notification, the same order as the writer */
    strcpy_s(blk->reply, sizeof(blk->reply), "pong from reader");
    MemoryBarrier();
    blk->reply_len = (uint32_t)strlen(blk->reply);
    if (!SetEvent(hRep)) {
        printf("SetEvent failed: %lu\n", GetLastError());
        goto cleanup;
    }
    rc = 0;

cleanup:
    if (hRep != NULL) CloseHandle(hRep);
    if (hReq != NULL) CloseHandle(hReq);
    if (blk  != NULL) UnmapViewOfFile(blk);
    if (hMap != NULL) CloseHandle(hMap);
    return rc;
}

MemoryBarrier is there to prevent the reordering that would let the length or a flag become visible before the payload has been fully written.5 If you want layout drift caught at build time, add /std:c11 to MSVC and pin sizeof(DemoBlock) with static_assert from <assert.h>.

Handling the same block from C# looks like this. The key point is that the offsets are spelled out as constants, written so that not a single byte drifts from the C struct. Named MemoryMappedFile and EventWaitHandle are Windows-only.1

// .NET 8 / Windows. Sender: dotnet run -- write, receiver: dotnet run -- read
using System.IO.MemoryMappedFiles;
using System.Text;

const string MapName = "Local\\KsShmDemo.v1.Block";
const string ReqName = "Local\\KsShmDemo.v1.Request";
const string RepName = "Local\\KsShmDemo.v1.Reply";
const uint Magic = 0x314D4853;   // 'S','H','M','1'
const uint Abi = 1;
const int BlockSize = 536;
const int MaxBody = 256;

// The same layout as the C DemoBlock, pinned down with offset constants
const int OffMagic = 0, OffAbi = 4, OffBlockSize = 8, OffState = 12;
const int OffRequestLen = 16, OffReplyLen = 20, OffRequest = 24, OffReply = 280;

bool isWriter = args.Length > 0 && args[0] == "write";

// The sender uses CreateNew. With CreateOrOpen, a second sender would simply
// open the block that is already in use and wreck the peer's exchange in the
// initialization below. CreateNew throws IOException when the name already
// exists, so you notice right there
// (same intent as the ERROR_ALREADY_EXISTS check on the C side)
using var mmf = isWriter
    ? MemoryMappedFile.CreateNew(MapName, BlockSize)
    : MemoryMappedFile.OpenExisting(MapName);
using var view = mmf.CreateViewAccessor(0, BlockSize);
using var reqEvent = new EventWaitHandle(false, EventResetMode.AutoReset, ReqName);
using var repEvent = new EventWaitHandle(false, EventResetMode.AutoReset, RepName);

if (isWriter)
{
    view.Write(OffMagic, Magic);
    view.Write(OffAbi, Abi);
    view.Write(OffBlockSize, (uint)BlockSize);
    Thread.MemoryBarrier();
    view.Write(OffState, 1u);              // READY

    byte[] request = Encoding.UTF8.GetBytes("ping from C#");
    view.WriteArray(OffRequest, request, 0, request.Length);
    Thread.MemoryBarrier();
    view.Write(OffRequestLen, (uint)request.Length);
    reqEvent.Set();

    if (!repEvent.WaitOne(TimeSpan.FromSeconds(5)))
    {
        Console.WriteLine("no response from the reader");
        return 1;
    }
    return PrintBody(OffReplyLen, OffReply, "reply");
}

if (!reqEvent.WaitOne(TimeSpan.FromSeconds(5)))
{
    Console.WriteLine("no request arrived");
    return 1;
}
if (view.ReadUInt32(OffMagic) != Magic || view.ReadUInt32(OffAbi) != Abi
    || view.ReadUInt32(OffBlockSize) != BlockSize || view.ReadUInt32(OffState) != 1u)
{
    Console.WriteLine("ABI mismatch, or initialization is not finished yet");
    return 1;
}
if (PrintBody(OffRequestLen, OffRequest, "request") != 0)
{
    return 1;
}

byte[] reply = Encoding.UTF8.GetBytes("pong from C#");
view.WriteArray(OffReply, reply, 0, reply.Length);
Thread.MemoryBarrier();
view.Write(OffReplyLen, (uint)reply.Length);
repEvent.Set();
return 0;

int PrintBody(int lenOffset, int bodyOffset, string label)
{
    uint length = view.ReadUInt32(lenOffset);
    if (length > MaxBody)
    {
        Console.WriteLine($"{label} length is out of range: {length}");
        return 1;
    }
    byte[] body = new byte[length];
    view.ReadArray(bodyOffset, body, 0, body.Length);
    Console.WriteLine($"{label}: {Encoding.UTF8.GetString(body)}");
    return 0;
}

The C version and the C# version use the same names and the same layout, so you can make either one the sender and the other the receiver and the round trip still works. That is what pinning an ABI means.

This sample deliberately does exactly one round trip. For continuous transfer, move on to the ring buffer in 6.4; to survive an abnormal writer exit, move on to the generation and heartbeat in 5.7.

7. What to Check on Windows vs. POSIX

Aspect Windows POSIX
Create / open CreateFileMapping / OpenFileMapping / MapViewOfFile6 shm_open / ftruncate / mmap3
Sharing without a disk file Pagefile-backed mapping created with INVALID_HANDLE_VALUE68 POSIX shared memory object + mmap3
Initial values Pagefile-backed pages are zero-initialized8 A new object has length 0. Newly allocated bytes are zero-initialized3
Synchronization mutex / semaphore / event / interlocked and the like25 Process-shared mutex / condvar / semaphore2018
Must not be used cross-process CRITICAL_SECTION, WaitOnAddress2110 A mutex / condvar left as PTHREAD_PROCESS_PRIVATE2019
Owner death WAIT_ABANDONED12 Robust mutex + EOWNERDEAD / pthread_mutex_consistent()1314
Deleting the name Disappears when the last handle / view is released28 shm_unlink removes the name. The object itself survives as long as references remain2223
Namespace / permissions Global\ / Local\, ACLs, SeCreateGlobalPrivilege1524 mode, umask, namespace, O_CREAT|O_EXCL3

C#’s MemoryMappedFile is essentially a wrapper over the Windows file mapping too. So:

  • open by the same name
  • use a separate mutex / event
  • read the view with an explicit layout
  • never place object references directly

— these basics remain exactly the same.1

MemoryMappedFile follows the same basicsDiagram showing that C# MemoryMappedFile is essentially a wrapper over the Windows file mapping, so the basics of opening by the same name, using a separate mutex or event, reading with an explicit layout, and not placing object references still hold.C# MemoryMappedFileA wrapper over file mappingOpen by the same nameSynchronize with a separate mutex / eventRead with an explicit layoutDo not place object references directly

Figure 25: With C#’s MemoryMappedFile, the same basics as file mapping carry over unchanged.

8. The Checklist to Run First

  • Do you really need shared memory? Is it large data on the same host?
  • Have you separated the control plane from the data plane?
  • Can the concurrency model be reduced to SPSC / 1 writer, many readers?
  • Does the leading header have magic / version / size / state / generation / heartbeat?
  • Are you placing any pointer / HANDLE / fd / STL object / std::mutex?
  • Is there a commit protocol so readers never see intermediate states?
  • Is exactly one initializer designated?
  • Is there a recovery procedure for abnormal termination?
  • Are names and permissions spelled out?
  • Is Global\ really necessary?
  • Are you assuming resize in place?
  • Have you tried writer kill / reader stall / version mismatch / insufficient permissions?

9. Summary

Used well, shared memory is genuinely powerful. Especially for

  • images
  • audio
  • sensor streams
  • large batches
  • high-frequency snapshots

and other large data within a single machine, it really pays off.

But the essence of shared memory is less speed than a transfer of responsibility. In exchange for fewer copies and less kernel-mediated messaging, you take on

  • synchronization
  • visibility
  • initialization
  • ABI
  • recovery
  • permissions
  • observability

yourself.

The essence of shared memory is a transfer of responsibilityDiagram showing the transfer of responsibility in which reducing copies and kernel-mediated messaging means the application takes on synchronization, visibility, initialization, ABI, recovery, permissions, and observability.Fewer copies and less messagingWhat you take on insteadSynchronization, visibility, initializationABI, recovery, permissionsObservability

Figure 26: The essence of shared memory is less speed than moving the responsibility for consistency onto you.

So for your first implementation, the safe shape is this.

  • An SPSC ring buffer or a double buffer
  • A fixed leading header
  • Offset references
  • Notification over a separate channel
  • With version / generation / heartbeat
  • With failure-path tests

Start from this shape and shared memory becomes a fairly well-behaved tool. Treat it from day one as fast common memory where anything goes, and over time it stops being an application and turns into archaeology.

10. References

  • Windows: file mapping and named shared memory basics682
  • Windows: namespace / security / synchronization1524512
  • POSIX: shm_open, shm_unlink, mmap, process-shared / robust synchronization322162013
  • .NET: MemoryMappedFile overview1

  1. Microsoft Learn, “Memory-Mapped Files” / Microsoft Learn, “MemoryMappedFile Class” ↩ ↩2 ↩3 ↩4

  2. Microsoft Learn, “Sharing Files and Memory” ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8

  3. man7.org, “shm_open(3)” ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10 ↩11

  4. Microsoft Learn, “/volatile (volatile Keyword Interpretation)” / Microsoft Learn, “volatile (C++)” ↩ ↩2

  5. Microsoft Learn, “Interlocked Variable Access” / Microsoft Learn, “MemoryBarrier function” ↩ ↩2 ↩3 ↩4 ↩5

  6. Microsoft Learn, “Creating Named Shared Memory” ↩ ↩2 ↩3 ↩4

  7. Microsoft Learn, “Scope of Allocated Memory” ↩ ↩2

  8. Microsoft Learn, “CreateFileMappingA function” ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9 ↩10

  9. man7.org, “POSIX Shared Memory” training slides ↩

  10. Microsoft Learn, “WaitOnAddress function” ↩ ↩2

  11. Microsoft Learn, “MapViewOfFileEx function” / Microsoft Learn, “MapViewOfFile function” ↩

  12. Microsoft Learn, “Mutex Objects” ↩ ↩2 ↩3

  13. man7.org, “pthread_mutex_lock(3p)” / man7.org, “pthread_mutexattr_setrobust(3)” ↩ ↩2 ↩3

  14. man7.org, “pthread_mutex_consistent(3)” / man7.org, “pthread_mutex_consistent(3p)” ↩ ↩2

  15. Microsoft Learn, “Kernel object namespaces” ↩ ↩2 ↩3 ↩4

  16. man7.org, “mmap(2)” ↩ ↩2

  17. Microsoft Learn, “Using Mutex Objects” ↩

  18. man7.org, “sem_init(3)” / man7.org, “sem_init(3p)” ↩ ↩2

  19. man7.org, “pthread_condattr_setpshared(3p)” / man7.org, “pthread_condattr_getpshared(3p)” ↩ ↩2

  20. man7.org, “pthread_mutexattr_getpshared(3)” / man7.org, “pthread_mutexattr_getpshared(3p)” ↩ ↩2 ↩3

  21. Microsoft Learn, “Critical Section Objects” ↩

  22. Microsoft Learn, “File Mapping Security and Access Rights” ↩ ↩2

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Windows App Development

Large-volume data exchange and process-isolation designs using shared memory, file mappings, and MemoryMappedFile are directly connected to Windows application development.

View Service Contact

Technical Consulting & Design Review

Design work that cuts down how much can go wrong — synchronization strategy, ABI design, recovery strategy, and separating the control plane from the data plane — is a good fit for technical consulting and design review.

View Service Contact

Frequently Asked Questions

Common questions about the topic of this article.

Can another process read a value written to shared memory correctly, right away?
Being visible and being safe to read are separate problems. Shared memory is a mechanism that shows the same byte sequence to multiple processes; it is not synchronization itself. Even when the writer intends to write length, then payload, then the ready flag in that order, a reader that reads with no synchronization at all can end up seeing the new length combined with the old payload. On both Windows and POSIX, access to shared memory assumes you combine it with a synchronization mechanism such as a mutex, a semaphore, or an event.
Can I place pointers, std::string, or HANDLEs in shared memory?
Better not to. Virtual addresses and process-local resources only have meaning inside that process's context, and mapping the same mapping in another process does not guarantee that the virtual addresses match. The same goes for std::vector, std::mutex, and CRITICAL_SECTION. If you need a reference, hold it as an offset from the base address, and keep the data you place in shared memory to fixed-width integers plus an explicit layout plus a versioned header.
Does volatile remove the need to synchronize shared memory?
No. volatile is not a magic spell that rescues a shared-memory design, and at minimum atomicity and mutual exclusion are separate problems. A design that busy-loops on a volatile bool wastes CPU, leaves the ordering guarantee between payload and ready flag vague, and easily picks up intermediate states. Windows's WaitOnAddress is also meant for threads within a single process, so it is safer not to treat it as a cross-process waiting mechanism. Move notification out to a primitive you can wait on, such as an event or a semaphore.
What should I decide first when designing shared memory?
Four things. Separating the control plane from the data plane, so control (start, stop, notification) goes over messaging while the data payload lives in shared memory; narrowing the concurrency model, where starting with an SPSC ring buffer or a double buffer is the hardest to get wrong; ownership and lifetime, meaning who creates it, who initializes it, who deletes it, and who recovers it; and the ABI design, including layout and versioning. Just putting magic, version, size, state, generation, and heartbeat in the leading header dramatically changes how easy incidents are to investigate.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.

Contact
Back to the Blog