Remove Duplicate Lines
Instantly remove duplicate lines from your text while preserving the original order or sorting the results alphabetically.
Related Tools
How to Use
- Paste Your List — Paste or type a list of lines — email addresses, keywords, data entries, or any text with one item per line.
- Choose Options — Choose case-sensitive or case-insensitive matching, whether to ignore leading and trailing spaces when comparing, and whether to keep the original order or sort the result A → Z or Z → A.
- Get Unique Lines — Click Remove Duplicates to replace your text with the deduplicated list, then copy or download it. Undo restores your original text if you change your mind.
About Remove Duplicate Lines
Remove Duplicate Lines keeps the first occurrence of every line and deletes the repeats that follow, leaving the unique lines in their original order or sorted. It is built for one-item-per-line data: email lists, keyword exports, URLs, tags, product codes, log messages and names copied out of a spreadsheet column. Live counters show how many lines, duplicates and unique lines your text has before you click, and Undo restores the original if the result is not what you wanted.
How matching works
- Lines are compared whole. With Case-sensitive matching on (the default), "Apple" and "apple" are different lines; switch it off to treat them as duplicates. Case is compared with simple lowercasing, so CAFÉ matches café, but café does not match cafe, and STRASSE does not match straße.
- With Ignore leading/trailing spaces when comparing on (the default), whitespace at either end of a line is ignored: spaces, tabs, non-breaking spaces and other Unicode spaces. The line that is kept is written out exactly as you typed it, original spacing included.
- The first occurrence wins and keeps its position. Order can then sort the unique lines A → Z or Z → A with language-aware sorting that ignores case and accents and puts numbers in natural order (item 2 before item 10). Lines that sort as equal keep their original order.
- Windows (CRLF) and old Mac (CR) line endings are recognized; the result uses LF line breaks. Copy and Download save the result as plain text.
Worked example
Input, one item per line: apple, Banana, apple, Apple, banana with a space on each side, cherry, apple (7 lines).
| Settings | Result | Removed |
|---|---|---|
| Defaults (case-sensitive, ignore spaces, original order) | apple, Banana, Apple, banana (spaces kept), cherry | 2 |
| Case-sensitive off | apple, Banana, cherry | 4 |
| Case-sensitive off, Sort Z → A | cherry, Banana, apple | 4 |
Sort A → Z on item 10, item 2, item 1 | item 1, item 2, item 10 | 0 |
Edge cases
- Blank lines are lines too. The first blank line is kept and later ones are removed as its duplicates; with spaces ignored, a line of only spaces counts as blank. To drop them all, run Remove Empty Lines first.
- Zero-width characters such as the zero-width space (U+200B) are not whitespace, so a line containing one will not match its twin. The Invisible Character Detector finds them.
- Text that looks identical can be stored differently: é typed as one character and é typed as e plus a combining accent are different lines here.
- Matching is per whole line, not per field. Two CSV rows that share an email address but differ in another column both stay; copy the column you care about into the tool on its own.
- A final line break counts as one more (empty) line in the totals.
When to use it
Merge two mailing lists and drop the overlap (turn case-sensitive matching off, since mail systems generally ignore letter case in addresses). Combine keyword exports from several sources before grouping them. Deduplicate a list of URLs before building a redirect map or a crawl list. Collapse repeated log lines to see the distinct messages. To count how often each word appears rather than remove repeats, use the Word Frequency Counter. The guide How to Remove Duplicate Lines From Text compares case-sensitive and case-insensitive matching and shows how to do the same in code.
Deduplication runs in your browser, so customer lists and exports stay on your device.
Frequently Asked Questions
Does the tool preserve the original line order?
Yes. By default, the tool keeps the first occurrence of each line in its original position and removes subsequent duplicates. You can optionally sort the output alphabetically (A → Z) or in reverse (Z → A) after deduplication; sorting is language-aware and puts numbers in natural order (2 before 10).
Is the duplicate check case-sensitive?
By default, matching is case-sensitive, so "Apple" and "apple" are treated as different lines. You can enable case-insensitive mode to treat them as duplicates, keeping only the first occurrence.
Does it ignore leading and trailing whitespace when comparing?
Yes, by default. With "Ignore leading/trailing spaces when comparing" switched on, spaces and tabs at the start and end of each line are ignored during comparison, so lines that differ only in surrounding whitespace count as duplicates. The lines you keep are output exactly as you typed them — trimming only affects the comparison. Switch the option off to treat such lines as different.
Can I see how many duplicates were removed?
After processing, the tool displays a summary showing the original line count, the number of duplicates found, and the final unique line count, giving you a clear picture of the cleanup.
What is the maximum number of lines the tool can process?
The tool handles tens of thousands of lines efficiently within your browser. Performance depends on your device, but typical lists of up to 100,000 lines are processed in under a second on modern hardware.
Can I use this to deduplicate email lists or CSV data?
Absolutely. Paste one email address or CSV row per line, and the tool removes exact duplicates. For CSV files where you need column-specific deduplication, extract the relevant column first, deduplicate it here, and then reconcile with your original dataset.