Text Cleaner

A list arrives from somewhere else — a spreadsheet column, a log, a form export — and it has padding, blank rows, repeats and no useful order. This page fixes all four in one pass, and it shows you the order it works in, because that order changes the answer.

The six steps, in the order they run

1. 1Trim each line. Leading and trailing spaces and tabs come off.
2. 2Collapse internal whitespace. Runs of spaces inside a line become one space.
3. 3Strip all whitespace. Every space inside a line disappears. This overrides step 2 when both are ticked.
4. 4Drop blank lines.
5. 5Remove duplicate lines, keeping the first occurrence.
6. 6Sort — A to Z, Z to A, shortest first, natural/numeric, or reverse the current order.

Turning any step off leaves the rest behaving identically. Nothing is conditional on anything else except step 3 overriding step 2.

Trimming before de-duplicating changes the answer

This is the part worth understanding, because it is where a list quietly ends up with the wrong number of rows in it.

Take six lines: ada@example.com, the same address with two spaces on each side, an empty line, ADA@example.com, and bob@example.com twice. With the default settings you get three lines out, two duplicates removed and one blank line dropped. Trimming happened first, so the padded copy became identical to the unpadded one and was caught.

Untick trimming and untick "ignore leading and trailing spaces when matching", and the same input gives four lines out and only one duplicate removed. The padded address is now a different string, and it stays.

Those two settings are not the same switch. Trimming edits the line, so the padding is gone from the output. Ignoring edge space only affects the comparison — the survivor keeps whatever spaces it arrived with. For a genuinely clean list, tick both.

Case is a separate decision again. ADA@example.com survives by default, because comparison is case-sensitive. Tick "ignore case" and you get two lines and three duplicates removed — usually right for email addresses, and wrong wherever case carries meaning.

Matching also normalises to Unicode NFC first, so an é typed as a single character and an é typed as e plus a combining accent are recognised as the same line. They look identical on screen and are not identical byte-for-byte, which is exactly the kind of duplicate that survives a naive pass.

Natural sort, and why file10 keeps jumping the queue

Alphabetical sorting compares character by character. file10 sorts before file2 because 1 comes before 2, and no amount of staring at the list makes that look correct. Natural sort reads runs of digits as numbers, so file1, file2, file10 land in the order you meant.

The sorting language selector matters for accented letters, and the two obvious choices genuinely disagree. Sorting apfel, äpfel, zebra under German gives apfel, äpfel, zebra, because ä files alongside a. Under Swedish it gives apfel, zebra, äpfel, because ä is its own letter at the end of the alphabet. Neither is a bug; pick the language your readers use.

What this will not do

It is a line tool and only a line tool. It will not split a comma-separated string into rows, and it has no idea what a CSV is — a quoted field containing a line break is read as two lines and may be sorted apart from its own record. Export to one item per line before you paste.

The counters read left to right in pipeline order too: blank lines are dropped before duplicates are counted, so three empty rows are reported as blank lines, never as duplicates.

Where to go from here

• Word Counter — counts, limits and reading time for the cleaned text.
• Case Converter — normalise casing before or after cleaning.
• Regex Tester — for the filtering rules this page deliberately does not attempt.