The short answer
Clean text in a fixed order, from the problems you can’t see to the ones you can:
- Keep a copy of the original.
- Remove invisible characters.
- Strip HTML and other markup.
- Fix line breaks.
- Normalize spaces.
- Remove empty lines.
- Normalize letter case, if case carries no meaning.
- Remove duplicates.
- Sort.
- Spot-check the result and compare the counts.
The order matters because each step hides problems from the later ones. Two lines that differ only by a trailing space, a capital letter or a zero-width character look identical to you, but a duplicate remover treats them as different. Every example below was run through the WritePadPro tools it mentions, and every tool runs in your browser, so the data you paste stays on your device.
Why order matters: one small example
Here is a list of four lines. The third ends with two spaces.
[email protected]
[email protected]
[email protected]␣␣
[email protected] With its default settings, Remove Duplicate Lines ignores leading and trailing spaces but matches case, so it removes only line 3 and keeps three lines. Untick “Case-sensitive matching”, or lowercase the list first, and line 2 goes too, leaving two addresses. Neither result is wrong. You decide whether “Anna” and “anna” are the same thing before you deduplicate, not after.
Step 1: Remove invisible characters
Text copied from websites, PDFs, chat apps and word processors can carry characters that take up no visible space:
- Zero-width space (U+200B) and zero-width joiners, often left by web pages and messaging apps.
- Non-breaking space (U+00A0), which looks like a space but doesn’t match one in searches or formulas.
- Byte order mark (U+FEFF), sometimes left at the start of files saved by older software.
- Soft hyphen (U+00AD), invisible unless a line breaks at that point.
- Bidirectional controls such as the right-to-left override (U+202E), which can make text display in a different order than it is stored.
They cause real damage. In this list, an invisible zero-width space sits inside Dana’s address (shown here as [U+200B]):
Anna Lee <[email protected]>
Ben Ortiz <[email protected]>, [email protected]
Dana Park <dana@exam[U+200B]ple.com>
Chen Wu <[email protected]>. Run it straight through the Email Extractor and you get three addresses: Dana is silently skipped, because the hidden character breaks the address. Clean it first with the Invisible Character Detector, which reports the zero-width space at line 3, column 21, and the extractor returns all four.
The detector highlights each hidden character with its name and position, then cleans the text: invisible characters are removed, space-like characters such as non-breaking spaces become ordinary spaces (so words never merge), and Unicode line separators become normal line breaks. Emoji sequences that need joiners are left intact.
Step 2: Strip HTML and markup
Copying from a page’s source, a CMS or an email template brings tags along. Before:
<h2>Opening hours</h2>
<p>Mon–Fri: <strong>9 am–5 pm</strong></p>
<ul><li>Closed on <a href="https://example.com/holidays">public holidays</a></li></ul>
<script>track();</script> After Strip HTML Tags, with “Keep line breaks” and “Keep link URLs” on:
Opening hours
Mon–Fri: 9 am–5 pm
Closed on public holidays (https://example.com/holidays) The tool parses the HTML the way a browser would, without running scripts or loading images. It decodes entities such as – and , drops the script together with its content, and turns paragraphs, headings and list items into separate lines. A quick find-and-replace of <[^>]+> (the Remove HTML tags preset in Find and Replace, or the editor’s Cleanup tab) is fine for simple fragments, but it leaves entities and script content behind.
Step 3: Fix line breaks
Text copied from a PDF keeps the PDF’s line ends and sometimes splits words with a hyphen:
The committee will review all appli-
cations received before the dead-
line and notify each applicant
by email.
Late applications are not accepted. Remove Line Breaks, with the default “keep paragraph breaks” mode and “Rejoin words split by a hyphen” turned on, gives:
The committee will review all applications received before the deadline and notify each applicant by email.
Late applications are not accepted. Without the rejoin option you get “appli-cations” and “dead-line”: the tool keeps the hyphen because it can’t know whether “well-known” or “appli-cations” was meant. Turn the option on for PDF text and leave it off for text with real hyphenated compounds at line ends. Smart mode, on by default, keeps list items (lines starting with -, *, • or 1.) on their own lines.
Step 4: Normalize spaces
Before (tabs, runs of spaces, a non-breaking space and trailing spaces):
Total:␣␣␣42 units[TAB](approx.)␣␣
␣␣␣␣Indented line
Price:[U+00A0]$10 After Remove Extra Spaces with all options on:
Total: 42 units (approx.)
Indented line
Price: $10 Untick “Trim spaces at line starts” to keep indentation, for code or poetry. The tool never touches line breaks, and it doesn’t remove zero-width characters, because they aren’t spaces; that is why invisible characters come first.
Step 5: Remove empty lines
Remove Empty Lines has three modes. Given a heading, three blank lines (one of them containing only spaces), a sentence and another heading:
- Remove all blank lines (default) deletes every empty or whitespace-only line, leaving three lines.
- Consolidate keeps exactly one blank line between blocks, which is what you want for paragraphs.
- Strict removes only lines with nothing on them and keeps the line of spaces, which helps when whitespace-only lines mean something in your format.
Step 6: Normalize letter case
Lowercase values where case carries no meaning: email addresses, tags, keywords, usernames and lookup keys. The Email Extractor has a “Lowercase emails” option, and the Case Converter handles whole texts (UPPERCASE, lowercase, Title Case, Sentence case, camelCase, snake_case and kebab-case).
Don’t lowercase names, product codes, passwords or anything where “ID” and “id” differ. If you only need case-insensitive matching, use the tool option instead of changing the data.
Step 7: Remove duplicates
Remove Duplicate Lines keeps the first occurrence of each line and outputs it exactly as written; the comparison options affect only matching. You can keep the original order or sort the result A to Z or Z to A.
- Case-sensitive matching (on by default): turn it off when “Paris” and “paris” are the same entry.
- Ignore leading and trailing spaces (on by default): catches lines that differ only by stray spaces.
Duplicates inside a line, such as repeated words in a sentence, are a different job; the Duplicate Word Finder handles those.
Step 8: Sort
Plain alphabetical sorting puts “10” before “2”. Sort Lines shows the difference:
| Input | A → Z | Natural |
|---|---|---|
| chapter-10.md | chapter-1.md | chapter-1.md |
| chapter-2.md | chapter-10.md | chapter-2.md |
| chapter-1.md | chapter-2.md | Chapter-3.md |
| Chapter-3.md | Chapter-3.md | chapter-10.md |
Sort Lines also sorts by length, by numeric value, at random or in reverse; sorts by a column with a comma, tab, semicolon, pipe or space separator (“Mouse,49 / Keyboard,129 / Cable,9” by column 2, numeric, gives Cable, Mouse, Keyboard); ignores accents or a leading “The/A/An” if you ask; and uses the alphabetical rules of the language you choose. It can remove duplicates, empty lines and stray spaces in the same pass.
Find and replace with regular expressions
When no single tool fits, Find and Replace applies several rules in one pass, with a live preview and undo. In regex mode, ^ and $ match the start and end of each line, $1 and $2 insert captured groups, and \n in the replacement inserts a line break. Matching ignores case unless you tick Match case. We tested each recipe below.
| Task | Find (regex) | Replace with | Example |
|---|---|---|---|
| Swap “Last, First” | ^(\w+), (\w+)$ | $2 $1 | Lee, Anna → Anna Lee |
| ISO date to US date | (\d{4})-(\d{2})-(\d{2}) | $2/$3/$1 | 2026-10-14 → 10/14/2026 |
| Remove list numbers | ^\d+[.)]\s* | (nothing) | 1. Apples → Apples |
| Remove citation marks | \s*\[\d+\] | (nothing) | long.[1] It → long. It |
| Lines into one comma list | \n | , | red / green / blue → red, green, blue |
| Collapse 3+ line breaks | \n{3,} | \n\n | Four breaks → one blank line |
| Trailing spaces | [ \t]+$ | (nothing) | “end␣␣␣” → “end” |
In JavaScript regular expressions, which the tool uses, \w matches only unaccented Latin letters, digits and underscores, so the name-swap recipe skips “Núñez, José”. The built-in “Last, First” → “First Last” preset uses Unicode letter classes and handles accented names. The other presets cover curly-to-straight quotes, multiple spaces, tabs, empty lines, wrapping lines in quotes, adding a “- ” prefix, and dashes and ellipses.
Extract what you need
Sometimes cleaning means keeping only one kind of data. The Email Extractor pulls addresses out of any text, HTML or CSV, and lists URLs and phone numbers on separate tabs. From the cleaned contact list in step 1, with “Lowercase emails” on and A → Z sorting, it returns:
[email protected]
[email protected]
[email protected]
[email protected] It removed the duplicate Anna (matching is case-insensitive by default), left the trailing period off Chen’s address and skips image names that look like addresses, such as [email protected]. It can also filter by domain, strip +tags (billing+invoices@ becomes billing@), join the list with commas or semicolons for an email client, and export a CSV.
Limits worth knowing
- Line tools aren’t CSV parsers. Sort Lines splits columns on the separator, so a quoted field that contains a comma is split too. For complex spreadsheets, clean in a spreadsheet app.
- Cleaning can remove meaning. Non-breaking spaces sometimes keep “10 kg” together on purpose; soft hyphens may be intentional in typeset text. Check before cleaning finished, formatted documents.
- Count before and after. Each tool reports what it removed. If 1,000 lines become 400, make sure you expected that.
- Keep the original. The tools have Undo, but a saved copy is the only undo that survives closing the tab.