When You Can't Avoid Building Your Own Logger: Practical Minimum Requirements and Integration Test Checks

bottom line first

The first goal of a custom logger is not rich features — it’s being trustworthy when things break. Here are the points to nail down in your first version.

• format is UTF-8 JSON Lines
• never break the one-record-per-line rule
• required fields: timestamp, level, category, message, fields, sessionId, processId
• default to one process, one file
• synchronous writes for low volume; single writer + bounded queue for higher volume
• Error / Critical and session start/end flush synchronously
• rotation and retention from v1
• when the destination is unavailable, do not silently fall back somewhere else

narrow the scope

The scope here is diagnostic logs used for triaging application failures. The priority is being able to trace, after the fact, when something happened, which operation it happened in, what happened, and what context surrounded it.

the minimum you actually need

1. format: UTF-8 JSON Lines

One record per line stays readable as plain text and is easy to parse later with scripts or tools. It’s also practical for incidents — if a write gets cut off mid-stream, you can identify exactly which line is broken.

2. fix the required fields

Seven required fields: timestamp, level, category, message, fields, sessionId, processId. A free-form message-only string log is hard to search. Adding too many fields, on the other hand, just burdens the call sites.

3. default to one process, one file

Letting multiple processes append to the same file makes exclusive locking, partial writes, rotation timing, and abnormal-termination handling all harder at once.

4. pick your write strategy by volume

• low volume → synchronous writes (simple, easy to investigate failures)
• high volume → single writer + bounded queue
• decide your queue-overflow policy up front (drop oldest / drop newest / surface a warning)

5. decide your flush rules

Flush synchronously on Error, Critical, and session start/end. Flushing on every routine Info line will slow you down.

6. rotation and retention from v1

It’s tempting to defer these as “we can add it later,” but they bite hard once you’re in production. At minimum, decide “this won’t grow forever” and “how many files do we keep.”

7. don’t silently fall back when the destination fails

If the configured destination is unavailable and you quietly write somewhere else, your operators lose time looking for logs that aren’t where they should be. Surface failures explicitly — via a notification, stderr, or similar.

common anti-patterns

• cramming everything into a message string
• sharing a single file across multiple processes
• going fully async without defining flush rules
• deferring rotation and retention
• silently writing to a different folder when the destination fails
• pulling network shipping or a local DB into v1

integration test checklist

A logger can’t be trusted on unit tests alone. You need integration tests against real files, real threads, and real processes. Six tests to pass at v1, at minimum:

1. normal writes from a single thread
2. concurrent writes from multiple threads (records must not get corrupted)
3. flush on Error / Critical (do they hit disk immediately?)
4. rotation and retention (does the trigger fire, and are files past the cap deleted?)
5. failure notification when the destination misbehaves (missing directory, insufficient permissions, disk full)
6. drain and final flush on normal shutdown

additional failure cases worth checking

• behavior when the destination directory does not exist
• behavior when write permission is missing
• the notification or return value when the disk is full
• behavior when the queue overflows
• whether stray newlines split a record across multiple lines
• whether shutdown logs still survive on near-exception exit paths

wrap-up

The first goal of a custom logger is not rich features — it’s being trustworthy when things break. Lock the format to UTF-8 JSON Lines, keep the required fields tight, default to one process per file, and decide your flush, rotation, retention, and failure behavior early. Then verify it with integration tests that use real files, real threads, and real processes. Before you grow the implementation, settle the minimum shape and the minimum test set first.