A spectre is haunting Unicode
- ID
- 14569
- Status
- summarized
- Published
- 15 Aug 2026, 10:34 PM
- Fetched
- 17 Aug 2026, 8:47 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://www.dampfkraft.com/ghost-characters.html
- Source URL
- https://hnrss.org/best
Summary
- Score
- 5.5
- Created
- 17 Aug 2026, 9:53 PM
- Tags
- Audience
- developersdatabase_learners
What happened
The article traces the origin of 'ghost characters' (幽霊文字) in JIS X 0208, the 1978 Japanese encoding standard that influenced Unicode. A 1997 investigation found most were accidental creations from manual cataloging errors—e.g., 妛 was born when a photocopy line between two pasted paper pieces was mistaken for a stroke. Only one character, 彁, remains entirely unexplained.
Why it matters
If you build systems handling CJK text, these ghost characters are permanently embedded in Unicode and can surface in user input, search indexing, or font rendering. Knowing they exist helps you avoid chasing phantom bugs when a character has no real-world usage or pronunciation.
Discussion angle
How encoding standards inherit human errors permanently—what does this imply for AI training data that scrapes Unicode text containing characters nobody actually uses?