Prompting Rules That Reduce Codex Mojibake on Windows

When you have Codex work with files containing Japanese text on Windows, the first thing that actually helps is not aligning every editor and shell setting. It is explicitly telling Codex how to read, how to write, and where to stop.

The situations that cause the most trouble look like this.

• UTF-8, CP932, and UTF-16-family files coexist
• The text looks readable on screen, but the interpretation of the actual bytes is off
• You only meant to tweak an existing file, but it gets re-saved in a different encoding
• Breakage happens in non-code files: CSV, TXT, logs, Markdown, configuration files
• A throwaway script or raw shell output gets saved as-is, and the corruption becomes permanent

OpenAI’s Codex tends to be more stable when you treat it less like a one-off chat partner and more like a teammate you use continuously, with settings and working rules in place. In particular, if your workflow has Codex read AGENTS.md, encoding rules belong there permanently rather than being repeated verbally every time.

This article organizes, for practical use, the instructions that pay off most when given up front so Codex can safely handle Japanese files on Windows.

flowchart TB
    accTitle: Fix the instructions before the settings
    accDescr: A diagram showing that when you have Codex handle Japanese files on Windows, what helps first is not aligning editor and shell settings but explicitly stating how to read, how to write, and where to stop.
    a1["Align editor and shell settings"] -.-> a2["Not what helps first"]
    a3["How to read"] --> a6["State it explicitly to Codex"]
    a4["How to write"] --> a6
    a5["Where to stop"] --> a6

Figure 1: What helps first is spelling out how to read, how to write, and where to stop, not the environment setup.

Background: the Codex setup and AGENTS.md this article assumes

For readers who do not use Codex, let me set out the background first.

Codex is a coding agent that actually reads and writes the files in your repository. It has several entry points, including a CLI that runs in your local terminal, editor extensions, and cloud-hosted variants, but this article assumes the usage where it edits the files in your local repository directly. Mojibake happens at the moment of editing and saving, so what matters is not which entry point you use but how you constrain the write path to the file.

AGENTS.md is a Markdown file where you keep the standing instructions for working in that repository. How it gets loaded has properties worth remembering.

• It does not live in one place. A global file such as ~/.codex/AGENTS.md in your home directory and the repository’s own AGENTS.md are both read.
• The files are concatenated in order from the repository root toward the working directory. That means an AGENTS.md placed in a subdirectory is concatenated later, so it carries more weight than the instructions above it.
• There is a cap on the total size. By default it is truncated at around 32 KiB, so a “write everything down for now” file loses its tail. Encoding rules are safer kept short and high up.

Because of those three properties, the practical approach is to write the encoding rules into the repository root AGENTS.md, briefly, and near the top. If a subdirectory carries a different write convention, that one wins.

flowchart TB
    accTitle: How AGENTS.md is loaded
    accDescr: A diagram showing that both the home-side and repository-side AGENTS.md files are read and concatenated from the root toward the working directory so that lower levels carry more weight, that the total is truncated at around 32 KiB by default, and that encoding rules therefore belong at the root, short and near the top.
    b1["AGENTS.md in the home directory"] --> b3["Concatenated from root toward working directory"]
    b2["AGENTS.md in the repository"] --> b3
    b3 --> b4["Lower levels, concatenated later, carry more weight"]
    b3 -.-> b5["Truncated at about 32 KiB by default"]
    b4 --> b6["Keep the rules at the root, short and near the top"]
    b5 -.-> b6

Figure 2: AGENTS.md has both a concatenation order and a size cap, so keep the rules short and high up at the root.

1. The Conclusion First

The single most effective way to reduce Codex mojibake on Windows is to fix the encoding work procedure in advance.

These are the rules that pay off the most.

• For existing files containing Japanese, have it check the likely encoding, BOM presence, and newline style before reading
• For files where mojibake is suspected, do not let it save until it is confident
• For existing files, have it preserve the original encoding, BOM, and newlines
• For new files, steer toward UTF-8 per repository convention
• For writes, allow only methods where the encoding can be made explicit
• After saving, have it re-read the file and verify representative Japanese lines

Put in short, day-to-day terms, it comes down to about this.

• Check before reading
• No saving when in doubt
• Preserve existing files, UTF-8 only for new ones
• Ban ambiguous write paths
• Re-read and verify at the end

flowchart TB
    accTitle: Five steps that prevent encoding corruption
    accDescr: A diagram showing in order the short day-to-day form of the procedure: check before reading, no saving when in doubt, preserve existing files and use UTF-8 only for new ones, ban ambiguous write paths, and re-read to verify at the end.
    c1["Check before reading"] --> c2["No saving when in doubt"]
    c2 --> c3["Preserve existing files, UTF-8 only for new ones"]
    c3 --> c4["Ban ambiguous write paths"]
    c4 --> c5["Re-read and verify at the end"]

Figure 3: The procedure worth fixing boils down to these five steps.

Conversely, these are the dangerous kinds of instructions.

• “Fix the mojibake”
• “Convert everything to UTF-8”
• “Output a CSV”
• “Just make it match”
• “Save it for now and let’s see”

None of these say at which point Codex should stop. For mojibake prevention, you have to specify not only what to do but where to stop short of saving.

Knowledge map for this article

When Codex is asked to handle Japanese files on Windows, the key is not to let it save a file on a guess. CP932 and UTF-8 cannot both interpret the same byte sequence correctly, so a procedure that checks the encoding candidates before reading, by looking for a BOM and by attempting a strict UTF-8 decode, reduces accidents in which the actual bytes are interpreted as something other than their original encoding. If a mismatch still slips through, the mojibake becomes fixed as corruption of the file itself, so it is recommended to keep a standing set of work rules in AGENTS.md: preserve the existing encoding, state the encoding explicitly when writing, and verify by reading the file back after saving. Windows PowerShell 5.1 and PowerShell 7 differ in their default write encoding, so avoiding ambiguous write paths is indispensable as well.

flowchart LR
    accTitle: Preventing Codex mojibake on Windows
    accDescr: Diagram showing how giving Codex character encoding work rules in AGENTS.md prevents encoding misinterpretation and the file corruption that comes from saving on a guess, together with the incompatibility between CP932 and UTF-8 and the difference in default encoding between PowerShell versions
    codex["Codex"]
    mojibake["Mojibake (Garbled Text)"]
    agents_md["AGENTS.md"]
    character_encoding_rules["Character Encoding Working Rules"]
    pre_read_encoding_check["Pre-Read Encoding and BOM Check"]
    unverified_save["Saving Without Confirming the Encoding"]
    existing_file_encoding_preservation["Preserving Existing File Encoding"]
    explicit_encoding_write["Writing with Explicit Encoding"]
    post_write_reread_verification["Post-Save Re-Read Verification"]
    ambiguous_write_path["Ambiguous Write Path"]
    utf8_migration_as_separate_task["UTF-8 Migration as a Separate Task"]
    cp932["CP932 (Windows-31J)"]
    utf_8["UTF-8"]
    bom["BOM (Byte Order Mark)"]
    strict_utf8_decode_check["Strict UTF-8 Decode Check"]
    encoding_misinterpretation["Encoding Misinterpretation"]
    reencoding_corruption["File Corruption from Wrong Encoding"]
    replacement_character["U+FFFD (Replacement Character)"]
    windows_powershell_5_1["Windows PowerShell 5.1"]
    ansi_codepage["ANSI Code Page"]
    utf_16["UTF-16"]
    powershell_7["PowerShell 7"]
    character_encoding_rules -->|"configured by"| agents_md
    character_encoding_rules -->|"recommended for"| codex
    pre_read_encoding_check -->|"recommended for"| codex
    unverified_save -->|"not recommended for"| codex
    existing_file_encoding_preservation -->|"recommended for"| codex
    explicit_encoding_write -->|"recommended for"| codex
    post_write_reread_verification -->|"recommended for"| codex
    ambiguous_write_path -->|"not recommended for"| codex
    utf8_migration_as_separate_task -->|"recommended for"| codex
    cp932 -.->|"incompatible with"| utf_8
    pre_read_encoding_check -->|"verified by"| bom
    pre_read_encoding_check -->|"verified by"| strict_utf8_decode_check
    encoding_misinterpretation -->|"may cause"| mojibake
    mojibake -.->|"may cause"| reencoding_corruption
    unverified_save -.->|"may cause"| reencoding_corruption
    pre_read_encoding_check -->|"mitigates"| encoding_misinterpretation
    reencoding_corruption -->|"verified by"| replacement_character
    windows_powershell_5_1 -->|"uses"| ansi_codepage
    windows_powershell_5_1 -->|"uses"| utf_16
    powershell_7 -->|"uses"| utf_8
    ambiguous_write_path -->|"uses"| windows_powershell_5_1
    ambiguous_write_path -->|"uses"| powershell_7

In the diagram a solid line marks a relation that always holds and a dashed line marks a conditional one (the conditions are given per relation on the detail page). The full list of relations (22 in total, with evidence and certainty) and the definitions of the main concepts are collected on the knowledge map detail page (in Japanese). Data: JSON-LD / Turtle

2. Why Mojibake Is So Common on Windows

The real problem is not that Codex is weak at Japanese. It is that on the Windows asset side, multiple encodings and multiple write paths coexist.

In practice, this kind of mixture is not unusual.

• Newer sources and Markdown are UTF-8
• Older CSVs, TXT files, logs, and configs are CP932-family
• Some outputs and tool-generated artifacts are UTF-16-family
• Save paths vary across editors, shells, and Excel-derived output
• Newlines are mixed between LF and CRLF as well

In this state, if Codex misinterprets the bytes even once, it can proceed to the next edit treating strings it failed to read as if they had been read correctly. And if it then saves, the problem is no longer a display issue. It becomes fixed as corruption of the file itself.

That is why mojibake prevention ultimately comes down to how you manage the I/O procedure.

flowchart TB
    accTitle: How mojibake becomes permanent corruption
    accDescr: A diagram showing that when Codex misinterprets an asset holding multiple encodings and write paths even once, it carries a string it never read into the next edit as if it had read it, and that saving turns the problem from a display issue into permanent corruption of the file itself.
    d1["Multiple encodings and write paths coexist"] --> d2["A single misinterpretation"]
    d2 --> d3["Moves to the next edit without having read it"]
    d3 --> d4["Saving locks it in as corruption"]
    d4 -.-> d5["The real countermeasure is managing the I/O procedure"]

Figure 4: Mojibake starts as a display problem and gets locked in as corruption the moment you save.

2.1 Four terms, up front

Let us settle the words that come up repeatedly from here on.

The important part is that neither CP932 nor UTF-8 is declared inside the file itself. Unless there is a BOM, a file does not announce how it should be read. That is why a check before reading is required.

flowchart TB
    accTitle: A file does not announce its own encoding
    accDescr: A diagram showing that whether a file is CP932 or UTF-8 is not recorded in the file itself, that without a BOM a file never announces how it should be read, and that a check before reading is therefore required.
    e1["A file holds only a byte sequence"] --> e2{"Is there a BOM"}
    e2 -->|"Yes"| e3["It is a Unicode encoding"]
    e2 -->|"No"| e4["UTF-8 or CP932 must be judged from the content"]
    e4 --> e5["So a check before reading is required"]

Figure 5: Without a BOM, a file will not tell you how to read it.

3. The Rules to Fix for Codex First

3.1 Have it check the likely encoding, BOM, and newlines before reading

The first rule is this.

Before reading an existing file that contains Japanese, check its likely encoding, BOM presence, and newline style, and if anything looks suspicious, do not proceed to interpreting the content as-is.

The point is to change the workflow to “before reading the text, first check the file’s basic facts: encoding, BOM, and newlines.”

How to have it actually check

If you only write “check it,” both Codex and humans will drift on method. It is steadier to bake the check procedure into the instruction itself. The following works on both Windows PowerShell 5.1 and PowerShell 7.

First, look at the leading bytes in hexadecimal. BOM presence is decided here.

$path  = 'C:\work\orders.csv'
$bytes = [System.IO.File]::ReadAllBytes($path)

# Print the first 16 bytes in hexadecimal
$head = $bytes[0..([Math]::Min(15, $bytes.Length - 1))]
($head | ForEach-Object { $_.ToString('X2') }) -join ' '

The blocks below reuse the $path and $bytes created here. Read the output like this.

Next, count the newline sequences. This is where “LF and CRLF are mixed” becomes visible.

$crlf = 0; $loneLf = 0; $loneCr = 0
for ($i = 0; $i -lt $bytes.Length; $i++) {
    if ($bytes[$i] -eq 0x0A) {
        if ($i -gt 0 -and $bytes[$i - 1] -eq 0x0D) { $crlf++ } else { $loneLf++ }
    }
    elseif ($bytes[$i] -eq 0x0D -and ($i -eq $bytes.Length - 1 -or $bytes[$i + 1] -ne 0x0A)) {
        $loneCr++
    }
}
"CRLF=$crlf  LF=$loneLf  CR=$loneCr"

Finally, narrow down the encoding candidates. When there is no BOM, whether the bytes decode strictly as UTF-8 is the quickest signal. UTF-8 puts strong constraints on the byte sequence, so reading a CP932 file strictly as UTF-8 usually fails partway through.

# The second argument $true tells it to throw on an invalid byte sequence
$strictUtf8 = New-Object System.Text.UTF8Encoding($false, $true)
try {
    $null = $strictUtf8.GetString($bytes)
    'Decoded cleanly as UTF-8'
}
catch {
    'Not UTF-8. Try candidates such as CP932'
}

When you try reading it as CP932, state the code page number explicitly.

# PowerShell 6.2 and later accept a code page number directly
Get-Content -Path $path -Encoding 932 -TotalCount 3

# In Windows PowerShell 5.1, Default is the system ANSI code page. On Japanese Windows that is CP932
Get-Content -Path $path -Encoding Default -TotalCount 3

A successful strict decode does not confirm UTF-8. An ASCII-only file passes either way. In the end, have it look at a representative Japanese line with its own eyes and confirm it reads correctly.

flowchart TB
    accTitle: The check procedure before reading
    accDescr: A diagram showing the procedure of viewing the leading bytes in hexadecimal to decide BOM presence, counting the newline sequences, narrowing the encoding candidates with a strict UTF-8 decode, and finally confirming by eye that a representative Japanese line reads correctly.
    f1["View the leading bytes in hexadecimal"] --> f2["Decide whether a BOM is present"]
    f2 --> f3["Count the newline sequences"]
    f3 --> f4["Narrow the candidates with a strict UTF-8 decode"]
    f4 --> f5["Confirm a representative Japanese line by eye"]
    f4 -.-> f6["An ASCII-only file passes either way"]

Figure 6: The pre-read check runs in this order: bytes, newlines, decode, then your own eyes.

3.2 Do not let it save a file with suspected mojibake on guesswork

This one is especially important.

When mojibake is suspected, treat the file as read-only during investigation and prohibit overwriting until the encoding interpretation is credible.

The same goes for humans: never save a file you have not actually been able to read. Saving on “it looks a bit broken, but this is probably it” turns that guess into permanent damage.

3.3 Preserve existing files and default to UTF-8 only for new ones

In the context of mojibake prevention, “unify everything to UTF-8” is surprisingly dangerous.

Eventually deciding to move the whole repo to UTF-8 is a legitimate call, but it is safer done as a separate task, while reviewing the diff and the blast radius. For everyday maintenance, this workflow is the stable one.

• When editing an existing file, preserve its original encoding
• When adding a new file, create it as UTF-8 per repo convention
• If an existing file needs conversion, keep that separate from ordinary functional fixes

3.4 Do not let it use ambiguous write paths by default

What makes things go wrong more often on Windows is “it’s just a small output, so write it sloppily from the shell.”

• Dumping output straight through a redirect
• Saving directly with a convenience command
• Promoting a temporary artifact straight into a production file

These paths often have no explicit encoding, which makes them a breeding ground for corruption. So it is safest to fix, for Codex, how the write mechanism itself is chosen.

The “default encoding” differs by PowerShell version

You will hit this one if you do not know it. PowerShell uses different default write encodings depending on the version.

In other words, the same script produces a different file depending on whether it ran under 5.1 or 7. This is the classic source of “it was fine on the dev machine, but it broke on the customer’s server.”

On top of that, 5.1 has the property that specifying a Unicode-family encoding always adds a BOM. There, -Encoding UTF8 means UTF-8 with BOM.

So make every write state its encoding.

# Define the variables so this section stands on its own
$path    = 'C:\work\orders.csv'
$newPath = 'C:\work\orders-new.csv'
$lines   = @('顧客コード,顧客名', 'C0001,株式会社サンプル')

# If the existing file is CP932, write it back as CP932 (PowerShell 6.2 and later)
Set-Content -Path $path -Value $lines -Encoding 932

# Match the system ANSI code page on Windows PowerShell 5.1
Set-Content -Path $path -Value $lines -Encoding Default

# Create a new file as UTF-8 without BOM (PowerShell 7)
Set-Content -Path $newPath -Value $lines -Encoding utf8NoBOM

If you want to steer the default across an entire session, $PSDefaultParameterValues is an option. But it applies only to that session, so a runbook that assumes “it is in our profile” will break in someone else’s environment.

$PSDefaultParameterValues['*:Encoding'] = 'utf8NoBOM'

Writing > instead of Out-File does not help either: since 5.1 it just calls Out-File internally, so the default encoding carries the same problem. Banning redirection and allowing only cmdlets or .NET APIs where the encoding can be written out is the most reliable approach.

flowchart TB
    accTitle: The same script produces different files
    accDescr: A diagram showing that because PowerShell default write encodings differ by version, the same script produces a different file depending on whether it ran under 5.1 or 7, and that banning redirection and allowing only cmdlets or .NET APIs where the encoding can be written out is the reliable approach.
    g0["The same script"] --> g1["Run under Windows PowerShell 5.1"]
    g0 --> g2["Run under PowerShell 7"]
    g1 --> g3["Files in different encodings result"]
    g2 --> g3
    g3 -.-> g4["Allow only write paths that state the encoding"]

Figure 7: Default encodings differ by version, so make every write state its encoding.

3.5 After saving, have it re-read and verify representative Japanese lines

“It saved successfully” and “it is not broken” are not the same thing.

What matters is having it read a representative Japanese line again after saving and check points like these.

• Whether replacement characters U+FFFD have crept in
• Whether ? has increased unnaturally
• Whether the diff has become a huge BOM-only or newline-only change
• Whether Japanese text that the work was not supposed to change is still intact

3.6 When warning signs appear, have it report before fixing

When encoding goes wrong, you limit the damage better by having it stop and report than by forcing a fix.

For example, if any of these signs appear, it is safer to treat the situation as abnormal for the moment.

• An increase in U+FFFD
• An increase in ?
• An unexpected BOM change
• A large newline-only diff
• Only the Japanese lines changing unnaturally and substantially

flowchart TB
    accTitle: Stop and report when warning signs appear
    accDescr: A diagram showing that when signs such as more replacement characters U+FFFD or question marks, an unexpected BOM change, or a large newline-only diff appear, having it stop and report limits the damage better than forcing a fix.
    h1["More U+FFFD or ? characters"] --> h4["Treat the situation as abnormal for now"]
    h2["An unexpected BOM change"] --> h4
    h3["A large newline-only diff"] --> h4
    h4 --> h5["Have it stop and report before fixing"]

Figure 8: When warning signs appear, have it stop and report rather than fix.

4. If You Hand It Over as a Short Instruction

For a short version to attach to each task, this much is already quite effective.

In this task, avoiding encoding corruption is the top priority.

- For existing files containing Japanese, check the likely encoding, BOM presence, and newline style before reading
- Do not save a file with suspected mojibake on guesswork
- Preserve the original encoding / BOM / newlines of existing files
- Create new files as UTF-8 per repo convention
- Use only write methods where the encoding can be made explicit
- After saving, re-read the file and confirm that representative Japanese lines are intact
- Report as abnormal any increase in `U+FFFD` or `?`, any BOM / newline damage, or any large diff

If the target files are already decided, adding this one line stabilizes things considerably.

Target files: <paths> / Representative strings: "<examples>"

Handing over representative strings pays off a lot. It gives Codex a concrete watch point: this Japanese text must not break.

flowchart TB
    accTitle: Give it a watch point with representative strings
    accDescr: A diagram showing that handing over the target files and the representative strings that must not break gives Codex a concrete watch point for Japanese text that must survive, which makes the instruction considerably steadier.
    i1["Name the target files"] --> i3["Give it a concrete watch point"]
    i2["Representative strings that must not break"] --> i3
    i3 --> i4["The instruction becomes considerably steadier"]

Figure 9: One line naming the target files and representative strings pins down what the verification should aim at.

5. A Template Worth Keeping in AGENTS.md

Rather than saying the same thing over and over, put it in AGENTS.md. Below is a practice-oriented template for repos that handle Japanese files on Windows.

# Text Encoding Rules

## Scope
This repository may contain Japanese text and mixed legacy encodings.
Avoid mojibake and accidental re-encoding above all else.

## Mandatory Rules
- Before reading or editing an existing text file that may contain Japanese, first determine:
  - likely encoding
  - BOM presence
  - newline style
- If mojibake is suspected, do not save the file until the encoding interpretation is credible.
- Preserve the original encoding, BOM, and newline style for existing files.
- Treat "convert to UTF-8" as a separate, explicit task.
- New files should follow repository convention. If there is no clear rule, prefer UTF-8 and state whether BOM is used.
- Do not use ambiguous write paths by default, such as shell redirection or convenience commands without explicit encoding control.
- After writing, reopen the file and verify representative Japanese lines.
- If any of the following appears, stop and report:
  - replacement characters
  - unexpected `?`
  - unintended BOM change
  - unintended newline conversion
  - whole-file diffs without a business reason

## Reporting Format
For each changed text file, report:
- path
- detected or preserved encoding
- BOM presence
- newline style
- how verification was performed
- whether representative Japanese text remained intact

The strength of this template is that it pins down not just how to edit but how not to break things. In particular, these two lines,

• If mojibake is suspected, do not save ...
• Treat "convert to UTF-8" as a separate, explicit task.

pull a lot of weight.

5.1 The Japanese-language version of the template

If your team’s reviews run in Japanese, a Japanese AGENTS.md can be easier to operate. The content is the same.

# 文字コードの取り扱い規約

## 適用範囲
このリポジトリには日本語テキストと、レガシーな文字コードのファイルが混在します。
文字化けと、意図しない再エンコードを、他の何よりも優先して避けてください。

## 必ず守ること
- 日本語を含む可能性がある既存テキストファイルは、読む前に次を確認する。
  - encoding の候補
  - BOM の有無
  - 改行コード
- 文字化けが疑われる間は、解釈に確信が持てるまでそのファイルを保存しない。
- 既存ファイルは、元の encoding、BOM、改行コードを維持する。
- 「UTF-8 に変換する」は、機能修正とは別の独立したタスクとして扱う。
- 新規ファイルはリポジトリ規約に従う。規約がなければ UTF-8 を選び、BOM の有無を明記する。
- encoding を明示できない書き込み経路を既定で使わない。
  シェルのリダイレクトや、encoding を指定できない便利コマンドが該当する。
- 書き込んだあとは開き直し、日本語の代表行が壊れていないことを確認する。
- 次のいずれかが出たら、修正しようとせず、いったん止めて報告する。
  - 置換文字 U+FFFD の増加
  - 想定していない `?` の増加
  - 意図しない BOM の変化
  - 意図しない改行コードの変換
  - 業務上の理由がないファイル全体の差分

## 報告のしかた
変更したテキストファイルごとに、次を報告する。
- パス
- 検出した、または維持した encoding
- BOM の有無
- 改行コード
- どうやって検証したか
- 日本語の代表文字列が無事だったか

Either the English or the Japanese version is enough on its own. Keeping both takes up twice as much of the size budget, so given the load-size cap on AGENTS.md, it is safer to settle on one.

6. Bad Instructions and Good Instructions

In mojibake prevention, the granularity of the instruction has a large say in the outcome.

The point is to always write in the check before touching anything and the verification after saving.

flowchart TB
    accTitle: The pattern for turning a bad instruction into a good one
    accDescr: A diagram showing that an instruction like fix the mojibake never says where to stop, and that adding the check before touching anything and the verification after saving turns it into an instruction that also covers where to stop short of saving.
    j1["Fix the mojibake"] --> j2["No stopping point is written down"]
    j2 -.->|"Add"| j3["The check before touching anything"]
    j2 -.->|"Add"| j4["The verification after saving"]
    j3 --> j5["An instruction that includes where to stop"]
    j4 --> j5

Figure 10: What separates a bad instruction from a good one is the check, the verification, and a stated stopping point.

7. A Checklist for Review Time

After Codex has done the work, fixing the checkpoints on the human side as well makes things steadier still.

• Is the encoding / BOM / newline handling reported for each changed file?
• Have only the Japanese lines changed unnaturally and substantially?
• Are there large newline-only diffs?
• Has U+FFFD or ? increased?
• Are there whole-file diffs unrelated to the business change?
• Have columns or quoting broken in CSVs or logs?

What matters in mojibake prevention is stopping suspicious diffs early, more than accumulating successful ones.

8. Summary

When you have Codex handle Japanese files on Windows, the first thing that helps is not perfecting the machine side. It is explicitly giving Codex the encoding work procedure.

Five points are worth remembering.

• Have it check encoding / BOM / newlines before reading
• If mojibake is suspected, do not let it save on guesswork
• Preserve existing files and steer only new files toward UTF-8
• Ban ambiguous write paths
• Have it re-read after saving and verify representative Japanese lines

And if you find yourself saying it every time, put it in AGENTS.md. That is the most practical move.

The core of mojibake prevention is not asking it to “handle Japanese properly.” It is writing down the conditions under which saving is allowed and the conditions under which it must stop. Write that much down and Codex becomes far easier to work with, even on Windows.

flowchart TB
    accTitle: The core is writing the conditions down
    accDescr: A diagram showing that the core of mojibake prevention is not asking the tool to handle Japanese properly but writing down the conditions under which saving is allowed and the conditions under which it must stop.
    k1["Asking it to handle Japanese properly"] -.-> k2["This is not the core"]
    k3["Writing down when saving is allowed"] --> k5["Codex becomes easy to work with even on Windows"]
    k4["Writing down when it must stop"] --> k5

Figure 11: The core is not how you ask but writing down the conditions for saving and for stopping.

9. References

• OpenAI Codex docs, Best practices
• OpenAI Codex docs, Custom instructions with AGENTS.md
• OpenAI Codex docs, Windows
• Microsoft Learn, about_Character_Encoding
• Microsoft Learn, Code Page Identifiers
• Microsoft Learn, Byte order mark

Related Articles

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

Related Topics

These topic pages place the article in a broader service and decision context.

Where This Topic Connects

This article connects naturally to the following service pages.

Frequently Asked Questions

Common questions about the topic of this article.

Why does Japanese text get garbled when you use Codex on Windows?

The real cause is not that Codex is weak at Japanese. It is that the Windows asset side holds several encodings at once, including UTF-8, CP932, and the UTF-16 family, along with several write paths. In that state, a single misinterpretation by Codex can lead it to carry a string it never actually read into the next edit as if it had read it correctly. Save at that point and the problem stops being a display issue and becomes permanent corruption of the file itself. That is why mojibake prevention ultimately comes down to how you manage the I/O procedure.

What instructions should you give to keep Codex from producing mojibake?

Fixing the encoding work procedure in advance is what helps most. Concretely, there are five points: have it check the likely encoding, BOM presence, and newline style before reading any existing file that contains Japanese; do not let it save a file with suspected mojibake until it is confident; preserve the original encoding of existing files and use UTF-8 only for new ones; allow only write methods where the encoding can be made explicit; and have it re-read the file after saving and verify representative Japanese lines. Handing it the target files plus the representative strings that must not break makes it steadier still.

Is it wrong to say 'fix the mojibake' or 'convert everything to UTF-8'?

Both are dangerous instructions. Neither says at which point Codex should stop short of saving, so it tends to save on guesswork and make the damage permanent. 'Convert everything to UTF-8' is especially risky: when editing existing files, have it preserve the original encoding, BOM, and newlines, and carve out a repository-wide move to UTF-8 as a separate task done while reviewing the diff and the blast radius. Instead, write instructions such as 'determine whether the file itself is corrupted or it is only a display-side issue, and do not save on guesswork,' covering both the check before you touch anything and the verification after saving.

Should encoding rules be written into AGENTS.md?

If you would otherwise repeat the same warnings in every task, keeping them permanently in AGENTS.md works better. Write down the pre-read check of encoding, BOM, and newlines, the ban on saving while mojibake is suspected, preservation of existing files, treating UTF-8 conversion as a separate task, the ban on ambiguous write paths, re-reading to verify after saving, and the rule to stop and report when something looks wrong. If you also fix the format for reporting the encoding, BOM, newline style, and verification method for each changed file, the review side becomes just as consistent.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.