Remove Duplicate Lines
Runs in your browser — your text is not sent anywhere to use this tool.
Save this tool
Keep Remove Duplicate Lines handy
Use your browser's Share menu, then “Add to Home Screen”.
Clean a list: drop repeats, blanks and stray spaces in one pass.
- Paste your list, one entry per line
- Choose whether capitalisation counts, and whether to sort the result
- Clean it, then copy or download the list
About this tool
Lists arrive full of repeats — email addresses pasted from three places, a log with the same error a hundred times, a keyword list assembled by hand. Removing the duplicates is trivial in a shell and awkward everywhere else, which is why people end up sorting a spreadsheet column and deleting rows by eye. The details are what make it reliable. Two lines that differ only by a trailing space are almost always the same line, and the difference is invisible, so trimming happens by default. Comparison ignores capitalisation unless you ask otherwise, because the same address written two ways is still one address. The original order is preserved: the first occurrence of each line survives, so the cleaned list can still be checked against what you started with. You can also invert the whole thing and see only the entries that repeated, which is the other half of the same question and normally means diffing two files by hand. Sorting understands accented letters, so a Danish list does not come back with æ, ø and å in a clump at the end.
Frequently asked questions
Does it keep my list in the original order?
Yes, unless you choose to sort it. The first occurrence of each line stays where it was and later copies are removed, so the result still lines up with the list you pasted. That makes it easy to check what was taken out.
Why are two lines that look identical not being matched?
Almost always trailing whitespace, which you cannot see. Keep the trim option on and it will not matter. The other cause is a difference in capitalisation combined with case sensitivity switched on.
Can I see which lines were duplicated instead of removing them?
Yes. Switch on the option to show only duplicates and you get one entry for each line that appeared more than once, in the order they were first seen. It is the fastest way to find the repeated addresses in a mailing list.
Should I keep the first or the last copy of a duplicate?
Keep the first if order matters, which is the default. Keep the last when later entries are more recent, such as a log where the newest line supersedes the earlier ones. Either way only one copy of each line survives.
Is my text uploaded anywhere?
No. The whole thing runs in your browser, so nothing is sent to a server. There is also a REST API for the same operation if you need to clean lists from a script, and that one necessarily processes on the server.
Help improve this tool
Found a bug, want a change, or need a tool we don't have?
You have an unsent draft.
Thanks — that's with us. You'll find the reply in your inbox.Go to my messages