Streams and framing — TCP's send and recv drift apart
What TCP delivers is a flow of bytes, not a sequence of messages. Send boundaries do not survive into recv — if you need boundaries, the application builds its own. Taught through a classic bug.
TCP is working correctly, yet the data "breaks"
Across five chapters we have watched TCP keep its three promises admirably. Yet in practice, defects like "received data occasionally goes missing" or "two messages arrive stuck together" are common on top of TCP. The packet capture shows no retransmissions, no loss. TCP is perfectly healthy. Still the application's data looks broken — why?
The answer lies in what TCP actually delivers. TCP promised "the same byte sequence, in the same order" — not "with the same boundaries." As Chapter 3 showed, TCP's protagonist is the byte, not the segment. Even if you call send() twice — 300 bytes then 200 bytes — they may merge into a single segment on their journey, or be recut into 120 + 380. The boundaries of the chunks that the receiver's recv() returns are unrelated to the sender's send boundaries.
This property is called a byte stream (no preservation of message boundaries). A socket is not a mail carrier delivering a bundle of letters; it is a seamless water pipe. Just as a pipe keeps no boundary between "the first glass of water" and "the second," neither does TCP.
Send boundaries (top row) are not inherited by recv boundaries (bottom row). Only byte content and order are promised. When '1 send = 1 recv' holds, it is a coincidence.
Two places where boundaries change
Where do boundaries actually change? There are two classic sites.
Both are correct optimizations as far as TCP is concerned. In development (local, low latency, small data), '1 send = 1 recv' keeps holding by luck, so bugs built on that assumption fire only in production, only occasionally. It is the birthplace of "unreproducible" defects.
Practice 6-1 — The byte-stream property
'Received data occasionally goes missing or sticks together' — let's uncover the real identity of this classic production bug.
Q1. On a TCP connection, the sender called send() twice (300 bytes, then 200 bytes). Which is a possible outcome of the receiver's recv() calls?
All TCP promises is 'the same byte sequence, in the same order' — send boundaries are not preserved. Depending on the path, the two sends may merge into one segment, or one send may split into several. The order of the 500 bytes is always preserved, so later data can never arrive first. Only the boundaries are untrustworthy.
Q2. Code that 'parses whatever recv() returned as one JSON message' works in development but occasionally fails in production. What is the most likely cause?
This is the classic of classics. Development environments are local and low-latency with small messages, so 1 send = 1 recv happens to hold. In production, the conditions for delay, splitting, and merging change, producing truncated JSON (parse errors) or two messages concatenated (the second half seems to vanish). Actual data corruption is prevented by TCP's checksum and retransmission, and ordering is always guaranteed. What is broken is not the data but the assumption about boundaries.
Q3. When small sends are made in rapid succession, TCP may bundle them into a single segment (the Nagle algorithm). Which correctly states this behavior's purpose and side effect?
Sending even 1 byte incurs roughly 40 bytes of TCP/IP headers. The Nagle algorithm suppresses this waste by holding back the next small write while small data is still awaiting an ACK, then sending them together. The price is latency on small interactive writes; where real-time behavior matters, it is disabled with TCP_NODELAY. Once again the key fact: TCP freely reshuffles the units you sent.
Framing — the application builds its own boundaries
If you need message boundaries, the only way is to embed the boundary information inside the byte sequence itself. This is called framing. There are two standards.
With either scheme, the receiver's code takes the same shape — append what recv delivered to a buffer, cut out only completed messages, and carry the remainder forward. The loop depends in no way on the number or boundaries of recv calls. Conversely, as long as this loop is correct, no boundary TCP chooses can break the application.
How to remember it: TCP has no concept of 'one message.' Inventing 'one message' is always the application's job. Even when you use an off-the-shelf protocol or library, this framing is at work underneath.
Practice 6-2 — Build your own boundaries (framing)
If boundaries are not preserved, the boundary information has to go inside the data itself.
Q4. In the length-prefix scheme (each message is sent with its body length prepended), which is the correct receiver implementation?
The receiver writes a loop that 'cuts messages out of the byte flow by itself' — (1) accumulate until the length field is complete, (2) read the length, (3) accumulate until the body reaches that length, (4) cut it out, process it, and carry the remainder forward. It depends in no way on the number or boundaries of recv calls. Time-based splitting breaks under varying delay, and TCP has no mechanism for application-level retransmission requests.
Q5. When using a delimiter such as a newline to mark message boundaries (e.g., one line = one message), what must you always design for?
If the delimiter itself appears in the body, the receiver cuts at a false boundary. So the delimiter scheme requires a guarantee that the body contains no delimiter — escaping (e.g., writing a newline as \n) or an encoding such as Base64. JSON Lines is a good example: it rides on the property that newlines inside JSON are always escaped as \n. Fixed-length padding or randomization would change the scheme into something else entirely — they do not address the problem.
Q6. Two messages, each a 4-byte length prefix + 300-byte body (304 bytes each), were sent back to back (608 bytes total). The receiver's first recv() returned 500 bytes. How many more bytes must be received to complete the second message?
The first message is bytes 1–304 and fits entirely inside the 500 bytes. The remaining 500 − 304 = 196 bytes are the head of the second message (4 prefix bytes + 192 body bytes). The second message is 304 bytes in total, so 304 − 196 = 108 bytes remain. Holding those 196 bytes in the buffer while waiting for the next recv — that is exactly the loop from the first question of Practice 6-2 in motion.
Key takeaways from this chapter
- TCP is a byte stream — the promise is 'the same bytes in the same order.' Send boundaries are not preserved
- Boundaries change routinely through merging (Nagle) and splitting (MSS). '1 send = 1 recv' is a development-environment coincidence
- If you need boundaries, use framing — a length prefix or a delimiter. Delimiters demand escaping by design
- The receiver-side pattern: 'accumulate, cut out only what is complete, carry the rest forward.' Never depend on recv boundaries
All the parts are now on the table. In the final chapter, we triage eight practice-flavored incident cases by translating each into 'which chapter is this about?'