Skip to content

Invisible Characters: How to Find and Remove Hidden Text

What Are Invisible Characters?

Invisible characters are Unicode characters that occupy space in a text string but produce no visible output on screen. They are real characters with real code points — they affect string length, break string comparisons, confuse search operations, and cause mysterious bugs — all while being completely invisible to anyone reading the text.

Detect every invisible character in your text with WritePadPro's Invisible Character Detector — paste any text and see each hidden character highlighted with its Unicode name and code point.

Invisible characters are not rare. They often turn up in text copied from a website, pasted from a word processor, exported from a PDF, or passed between systems. A single invisible character can break a database query, fail a password comparison, corrupt a CSV import, or cause two seemingly identical strings to not match. Understanding what they are, where they come from, and how to detect them helps anyone who works with text data.

The Complete Invisible Character Reference

Here are the most common invisible characters you will encounter, organized by how frequently they cause problems.

High-Frequency Problem Characters

CharacterUnicodeNameCommon Source
(invisible)U+00A0Non-Breaking Space (NBSP)Word processors, HTML ( ), PDFs, macOS Option+Space
(invisible)U+200BZero-Width Space (ZWSP)Web content, CMS editors, Unicode text processing
(invisible)U+FEFFByte Order Mark (BOM)UTF-8 files saved by Windows Notepad, Excel CSV exports
(invisible)U+00ADSoft HyphenWord processors (auto-hyphenation), web content
(invisible)U+200CZero-Width Non-Joiner (ZWNJ)Persian, Arabic, Indic scripts; web content
(invisible)U+200DZero-Width Joiner (ZWJ)Emoji sequences (family emoji), Indic scripts

Direction and Formatting Characters

CharacterUnicodeNamePurpose
(invisible)U+200ELeft-to-Right Mark (LRM)Forces LTR direction in bidirectional text
(invisible)U+200FRight-to-Left Mark (RLM)Forces RTL direction in bidirectional text
(invisible)U+2060Word JoinerPrevents line breaks (like NBSP but zero-width)
(invisible)U+2028Line SeparatorUnicode line break (different from \n)
(invisible)U+2029Paragraph SeparatorUnicode paragraph break (different from \n\n)
(invisible)U+180EMongolian Vowel SeparatorMongolian text; sometimes misused as whitespace

Control Characters

RangeUnicodeNameNotes
C0 controlsU+0000 - U+001FASCII control charactersIncludes tab (09), LF (0A), CR (0D). Others are invisible and usually problematic.
DeleteU+007FDELLegacy control character, invisible
C1 controlsU+0080 - U+009FLatin-1 control charactersRarely used intentionally; often appear from encoding errors

For related whitespace issues, see our Remove Extra Spaces Guide. For HTML-specific invisible elements, see the Strip HTML Tags tool.

Where Invisible Characters Come From

Copy-Paste From Websites

Web pages contain invisible characters for legitimate purposes: non-breaking spaces between words that should not be separated, zero-width spaces as word-break hints for long URLs, direction marks for multilingual content, and soft hyphens for responsive hyphenation. When you copy text from a web page, all of these invisible characters come with it.

Word Processors (Microsoft Word, Google Docs)

Word processors can carry non-breaking spaces (often typed on purpose, for example between "Mr." and a surname), optional hyphens at hyphenation points, and sometimes zero-width characters. These are invisible in the word processor but become visible problems when the text is pasted into code, databases, or other systems.

PDF Text Extraction

PDFs use a variety of invisible characters for layout purposes. Text copied from PDFs frequently contains non-breaking spaces (instead of regular spaces), soft hyphens (from line-end hyphenation), and sometimes other invisible characters, depending on how the PDF was produced.

UTF-8 BOM From Windows

Windows Notepad (before 2019) and some Windows applications prepend a Byte Order Mark (BOM) character (U+FEFF) to the beginning of UTF-8 files. This invisible character at the start of a file can break scripts that parse the first line, corrupt JSON/XML/CSV parsing, and cause "unexpected character" errors in programming. Since 2019, Notepad saves UTF-8 without a BOM by default, but legacy files and some exports still carry one.

Database and API Responses

Data that passes through several systems can pick up invisible characters along the way — for example, a BOM from a CSV export, non-breaking spaces pasted into a web form, or direction marks from multilingual input. For stripping HTML-sourced invisible characters, see our Strip HTML Tags Guide.

Problems Invisible Characters Cause

Broken String Comparisons

The string "hello" (5 characters) and "hello" with a trailing zero-width space (6 characters) are not equal in any programming language. This breaks password comparisons (user cannot log in despite typing the correct password), dictionary lookups (word not found despite being spelled correctly), data deduplication (identical-looking records treated as unique), and conditional logic (if statements that should match but do not).

Failed Searches

Searching for "hello" in a document will not find "hel<ZWSP>lo" (hello with a zero-width space in the middle). The search function sees them as different strings. This can affect browser Find (Ctrl+F), code editor search, database queries (WHERE name = 'hello'), and spreadsheet VLOOKUP/INDEX functions.

Display Glitches

Some invisible characters affect rendering: direction marks (LRM, RLM) can change the order in which nearby punctuation and words appear. Soft hyphens display a hyphen at line-break points but nothing otherwise — causing words to appear hyphenated in some contexts and unhyphenated in others. BOM characters at the start of included files can produce mystery blank spaces or "invalid character" symbols.

Security Vulnerabilities

Invisible characters have been used in security attacks. Lookalike letters (homograph attacks) and zero-width characters can make two different strings — usernames, links, keywords — look identical. The right-to-left override character (U+202E) can disguise a file’s real extension: a file named "invoice[U+202E]fdp.exe" displays as "invoiceexe.pdf". In 2021, researchers demonstrated "Trojan Source" attacks, in which bidirectional control characters make source code display differently from what the compiler sees.

Data Corruption

CSV files with invisible characters in field values produce phantom columns or broken parsing. JSON with BOM fails strict parsers. XML with unexpected control characters is rejected. YAML with invisible whitespace produces incorrect indentation interpretation. For related line break issues in data, see our Remove Line Breaks Guide.

How to Detect Invisible Characters

WritePadPro's Invisible Character Detector

The most straightforward method: paste your text into WritePadPro's Invisible Character Detector. The tool scans every character and highlights invisible ones with their Unicode name and code point. You see exactly what is hiding in your text without needing any technical knowledge.

Text Editor Methods

  • VS Code: Enable "Render Whitespace: all" in settings. Install the "Gremlins" extension to highlight invisible characters with red backgrounds.
  • Sublime Text: Set "draw_white_space": "all" in your settings to show spaces and tabs; many control and zero-width characters are drawn as code-point placeholders.
  • Notepad++: View > Show Symbol > Show All Characters displays every invisible character with a distinct symbol.

Command Line

# Show hex dump of file (reveals all bytes)
xxd filename.txt | head

# Show non-ASCII characters
grep -P "[^\x00-\x7F]" filename.txt

# Find zero-width spaces specifically
grep -P "\x{200B}" filename.txt

JavaScript Detection

// Detect common invisible characters
const invisibles = /[\u200B\u200C\u200D\u200E\u200F\uFEFF\u00AD\u2060\u2028\u2029]/g;
const found = text.match(invisibles);
console.log(found ? found.length + " invisible chars found" : "Clean");

How to Remove Invisible Characters

Remove All Invisible Characters

// JavaScript: remove all common invisible characters
text = text.replace(/[\u200B-\u200D\u200E\u200F\uFEFF\u00AD\u2060\u2028\u2029\u180E]/g, "");

// Also remove BOM from start of string
text = text.replace(/^\uFEFF/, "");

This simple version also deletes zero-width joiners inside emoji, which splits sequences such as family emoji, and deletes line and paragraph separators instead of turning them into line breaks. WritePadPro’s detector handles both cases.

Replace NBSP With Regular Spaces

// Non-breaking spaces often should become regular spaces, not be deleted
text = text.replace(/\u00A0/g, " ");

The Safe Cleanup Pipeline

  1. Replace non-breaking spaces with regular spaces (preserve word separation)
  2. Remove zero-width characters (ZWSP, ZWNJ, ZWJ, word joiner)
  3. Remove BOM characters
  4. Remove soft hyphens
  5. Remove direction marks (unless text is intentionally bidirectional)
  6. Normalize whitespace with Remove Extra Spaces

The order matters — replacing NBSP before normalizing whitespace prevents double-space artifacts.

Using WritePadPro's Invisible Character Detector

WritePadPro's Invisible Character Detector reveals every hidden character in your text.

Step 1: Paste Your Text

Open the Invisible Character Detector and paste your suspect text — from web pages, documents, databases, emails, or any source where invisible characters might hide.

Step 2: Scan

Click Detect Characters. The tool checks every code point against its list of invisible and special characters: zero-width and format characters (ZWSP, ZWNJ, ZWJ, word joiner, BOM, soft hyphen and others), bidirectional controls (LRM, RLM, embeddings, overrides and isolates), C0 and C1 control characters (except tab, line feed and carriage return), variation selectors, invisible tag characters, special spaces such as NBSP and en, em, thin and ideographic spaces, and Unicode line and paragraph separators. Joiners and selectors that belong to an emoji are recognized and left alone.

Step 3: Review the Results

The results table lists each character type found with:

  • Character name — human-readable name (e.g., "Zero-Width Space")
  • Code point — the exact identifier (e.g., U+200B)
  • Count — how many times it appears
  • What it does and Cleaning — its effect and what cleaning will do with it
  • First positions (line:col) — where the first occurrences sit

Below the table, your text is shown with each hidden character replaced by a highlighted [U+XXXX] marker, so you can see it in context.

Step 4: Clean Up

Based on the detection results, decide which invisible characters to remove and which to keep (some, like non-breaking spaces between "Mr." and a name, may be intentional). Untick any type you want to keep, then click Clean selected characters. Invisible characters are removed, special spaces become normal spaces so words never run together, and line or paragraph separators become line breaks. Then copy or download the cleaned text, or use Undo cleaning. Copy report copies the findings as a list.

Privacy

All detection runs locally in your browser. Your text — including potentially sensitive content where invisible characters might hide (passwords, database fields, API keys) — is never transmitted to any server.

Summary

Invisible characters are real Unicode characters that produce no visible output but affect string length, comparisons, searches, rendering, and security. The most common culprits are non-breaking spaces (U+00A0), zero-width spaces (U+200B), byte order marks (U+FEFF), and soft hyphens (U+00AD). They enter your text through copy-paste from websites, word processors, PDFs, and multi-system data pipelines.

The problems they cause range from annoying (broken Ctrl+F searches) to critical (failed authentication, data corruption, security vulnerabilities). Detection requires specialized tools because, by definition, you cannot see these characters by looking at the text.

Detect invisible characters with WritePadPro's Invisible Character Detector — each hidden character it finds is listed with its Unicode name and position and can be cleaned in one click, all in your browser.

Frequently Asked Questions

What is a zero-width space and where does it come from?

A zero-width space (Unicode U+200B) is a character that occupies zero visual width — it is completely invisible but is a real character that affects string length and comparisons. It was designed as a word-break hint: placed inside a long string (like a URL), it tells the text renderer that a line break is allowed at that point. Web content management systems, rich text editors, and some Unicode text processing tools insert zero-width spaces automatically. They commonly appear in text copied from websites, CMS platforms (WordPress, Drupal), and chat applications. They cause problems because two strings that look identical may not match if one contains hidden zero-width spaces.

What is BOM and why does it cause problems?

BOM (Byte Order Mark, Unicode U+FEFF) is a special character placed at the beginning of a file to indicate the byte order (endianness) of the encoding. In UTF-8, BOM is technically unnecessary because UTF-8 has no byte order ambiguity — but some Windows applications (such as older versions of Notepad, and Excel’s "CSV UTF-8" export) prepend it anyway. The BOM character is invisible but occupies 3 bytes at the start of the file. It causes problems because parsers reading the first line of the file encounter an unexpected character: JSON parsers fail, XML parsers report errors, CSV importers create a phantom first column, and PHP scripts may output whitespace before headers.

Can invisible characters be used for security attacks?

Yes. Several attack vectors exploit invisible characters. Homograph attacks use lookalike characters, and sometimes zero-width characters, to make two different URLs, usernames or domain names look identical. The right-to-left override character (U+202E) can reverse part of a filename's display — a file named 'invoice[U+202E]fdp.exe' shows as 'invoiceexe.pdf', disguising an executable. The 'Trojan Source' attack (2021) demonstrated that bidirectional override characters in source code can make code display differently from how the compiler interprets it, allowing malicious logic to be hidden in plain sight during code review. Always scan user input and imported text for invisible characters in security-sensitive contexts.

How do I know if my text contains invisible characters?

You cannot detect invisible characters by visual inspection — that is what makes them invisible. Three detection methods exist: (1) Use WritePadPro's Invisible Character Detector, which scans every character and highlights hidden ones with their Unicode name and position. (2) Use a code editor with invisible character visualization enabled — VS Code with the 'Gremlins' extension or Notepad++ with 'Show All Characters.' (3) Check string length: if a string that appears to be 5 characters reports a length of 6 or 7, it contains invisible characters. In JavaScript, comparing text.length with the number of characters you can see reveals the difference.

Should I always remove invisible characters?

Not always. Some invisible characters serve legitimate purposes. Non-breaking spaces between 'Mr.' and a surname prevent awkward line breaks. Zero-width joiners (ZWJ) in emoji sequences create compound emoji (the family emoji is multiple person emoji joined by ZWJ). Direction marks are essential for correctly displaying bidirectional text (mixing English and Arabic). Soft hyphens enable proper responsive hyphenation in web typography. Remove invisible characters when they cause problems (broken comparisons, failed searches, data corruption). Keep them when they serve a specific formatting or linguistic purpose. WritePadPro's detector helps you make this decision by identifying each character and its type; you can untick the types you want to keep before cleaning, and joiners that belong to emoji are left alone automatically.

Try These Tools

Related Articles