Remove Duplicate Lines
Paste a list and keep each line only once, in its original order or sorted.
Remove Duplicate Lines Online
Email lists merged from two exports, keywords collected from several tools, log lines, file names, tags: lists pick up duplicates all the time. This page opens the Remove duplicate lines tab of the Word Counter, which keeps every line once and tells you how many it removed.
Options
| Option | Effect |
|---|---|
| Ignore case | Apple, apple and APPLE count as the same line |
| Trim spaces | Spaces at the start and end of a line are ignored and removed |
| Remove empty lines | Blank lines are dropped instead of kept once |
| Sort | Keep the original order, or sort A to Z or Z to A |
Doing it on the command line
On Linux and macOS, sort -u list.txt removes duplicates but also sorts the list. To keep the original order, awk '!seen[$0]++' list.txt is the classic one-liner. In PowerShell, Get-Content list.txt | Select-Object -Unique keeps the order too, and like the others treats Apple and apple as different lines. This page does the same without a terminal and with the options above.
Related tools
To compare two versions of a list rather than clean one, use Text Compare. To change the case of every line, use the Case Converter.