Does it keep the original order?

By default yes — the first occurrence of each line stays where it was and later repeats are dropped. Turn on sorting if you would rather have the output alphabetised.

What counts as a duplicate?

An exact character-for-character match, unless you turn on “ignore case” or “trim whitespace”. Those two options catch the overwhelming majority of near-duplicates in real lists, where the difference is a trailing space or a capital letter.

How large a list can it handle?

Comfortably into the hundreds of thousands of lines. Deduplication uses a hash set, so it stays fast as the list grows; the limit is how much text your browser will hold in a textarea.

Remove Duplicate Lines

Paste a list and every repeated line is removed, leaving one of each. You can ignore case, trim surrounding whitespace, and choose whether to keep the original order or sort the result. It handles hundreds of thousands of lines in the browser, with nothing uploaded.

Runs in your browser — nothing is uploaded

Questions

Does it keep the original order?: By default yes — the first occurrence of each line stays where it was and later repeats are dropped. Turn on sorting if you would rather have the output alphabetised.
What counts as a duplicate?: An exact character-for-character match, unless you turn on “ignore case” or “trim whitespace”. Those two options catch the overwhelming majority of near-duplicates in real lists, where the difference is a trailing space or a capital letter.
How large a list can it handle?: Comfortably into the hundreds of thousands of lines. Deduplication uses a hash set, so it stays fast as the list grows; the limit is how much text your browser will hold in a textarea.

Remove Duplicate Lines

Questions

What people use next