How the duplicate line remover works
Repeated rows can break imports, inflate list counts, and create duplicate work. This cleaner makes its matching rules explicit while preserving the original text of the occurrence you choose to keep.
Calculation method
The tool splits the input at line breaks, optionally trims surrounding whitespace for comparison, optionally folds case, and stores the first or last occurrence of each comparison key. Blank lines can be removed independently.
Comparison key = optional case-fold(optional trim(line)); output = one selected occurrence per unique key.Assumptions and counting rules
- Whitespace trimming changes the comparison key but not the text of a retained line.
- Case-insensitive matching uses the browser's Canadian English locale rules.
- The duplicate audit is capped at 50 displayed rows, while the cleaned output includes the full result.
Practical examples
Remove exact repeats
A, B, A becomes A, B when case-sensitive matching is on.
The retained values keep their original order.
Ignore case
Maple and maple are treated as the same key when case sensitivity is off.
Choose first or last to decide which spelling remains.
Ignore surrounding spaces
Item and two spaces followed by Item can match while the retained line keeps its original text.
Useful when pasted data has accidental padding.
Keep the final spelling
Red, red, RED becomes RED when case is ignored and Last occurrence is selected.
The selected original line is moved nowhere; earlier matching rows are removed.
Audit removed rows
A five-line input with two repeats reports three unique output lines and two removed rows.
The audit helps verify what changed before an import or handoff.
Common mistakes
- Removing duplicates before deciding whether case differences are meaningful.
- Assuming whitespace was rewritten when it was only ignored for comparison.
- Deleting blank records when row position carries meaning in another system.
Edge cases
- An input ending in a line break contains a final blank line.
- Blank lines are duplicates when blank removal is off.
- Visually similar Unicode characters may still be different underlying text.
Questions specific to this tool
Does trimming change my output?
No. It changes how lines are compared; the selected original line is retained.
Can I keep the last copy?
Yes. Choose Last occurrence and the final matching line remains in its original position.
Is the pasted list uploaded?
No. Processing runs locally in the browser and the list is not intentionally saved.
Last reviewed: August 26, 2026 · Method version 1.0.0