The online deduplication tool removes duplicate lines. Put the list content into the text box, one item per line, then click the deduplicate button to get a list with no duplicate data. It can deduplicate txt text, js, java arrays and .net arrays.
This tool belongs to the “Text processing” category.
Your input is processed locally in the browser whenever possible. We do not store it.
Chinese is counted per character; English is reported both per character and per whitespace-separated word.
Bounded by the browser and the request size. Tens of thousands of characters are routine; split a whole book into chunks.
Many characters map one-to-many and need context. The tool uses common mappings — proof-read proper nouns.